Neuroplasticity games for depression

ABSTRACT

A training program is configured to systematically drive neurological changes to treat depression, mood and anxiety disorders. The training program includes an inference renormalization game that presents three subsets of stimuli and prompts a game participant to selectively respond or withhold responding to one or two of the stimulus subsets.

CROSS REFERENCE TO RELATED APPLICATION(S)

This application is a continuation of U.S. patent application Ser. No.14/201,689, filed Mar. 7, 2014, which is herein incorporated byreference for all purposes.

This application also claims the benefit of the following U.S.Provisional Patent Applications, which are incorporated herein in theirentirety for all purposes:

Ser. No. Filing Date: Title: 61/774,378 Mar. 7, 2013 Addiction61/774,390 Mar. 7, 2013 Overcoming Attention Deficit and HyperactivityDisorder 61/774,400 Mar. 7, 2013 Increasing Resilience AgainstAlzheimer's Disease 61/774,413 Mar. 7, 2013 Autism 61/774,420 Mar. 7,2013 Major Depressive Disorder 61/777,066 Mar. 12, 2013 Software-BasedTraining to Track and Modify Cognitive Behavior and Emotion throughAttention and Stimulus Valuation 61/777,080 Mar. 12, 2013 TraumaticBrain Injury 61/779,024 Mar. 13, 2013 Program to Ameliorate theNeurological Distortions Arising from a History of Childhood Stress orAbuse 61/782,571 Mar. 14, 2013 Novel Strategy to Accelerate Recovery ofSpeech Understanding in Individuals with Cochlear Implants or HearingAids 61/780,495 Mar. 13, 2013 New Tool Designed to Establish or RestoreSocial Cognition and Social Control Abilities in Individuals Which TheyHave Not Developed Normally, or Have been Degraded or Lost Due toIllness

FIELD OF THE INVENTION

This invention relates in general to amusement devices, and moreparticularly, to games that not only entertain but also treat cognitiveimpairments.

BACKGROUND OF THE INVENTION

Depression and Mood and Anxiety Disorders.

Major depressive disorder (MDD) is a common, recurrent and disablingcondition marked by significant impairments in social and occupationalfunctioning. MDD is the third leading cause of global disease burden,with annual costs exceeding $50 billion in the U.S. workplace alone. Thelifetime prevalence of depression is 17% in the U.S., where more than1.5 million years/annum are lost to MDD-related disability. Suicide isthe 10th leading cause of death in the U.S., and two-thirds of allsuicides are committed by individuals with MDD. Despite the availabilityof a variety of medical approaches, up to 50% of patients do not respondto psychological or pharmacological treatments. In the U.S., standardcare results in remission (complete recovery) in only one of three MDDpatients. Nearly 40% of patients who do recover relapse back into MDDwithin two years. This occurs in part because many patients voluntarilyabandon antidepressant treatment, partly due to what for them areintolerable side effects. In light of these staggering statistics, it isclear that many unmet needs remain in the treatment and relapseprevention of MDD.

MDD and mood and anxiety disorders (MA) are associated with deficits inattention, executive function, learning and memory. Individuals with MDDor MA are particularly impaired at inhibiting or disengaging attentionfrom negative information, and amplify the significance of personalfailure by committing more errors immediately after mistakes. It hasbeen suggested that these deficits play a key role in the emergence andmaintenance of negative processing biasing, which has been implicated inthe etiology of depression and MA. In addition, broadly generalizeddeficits in processing speed contribute to these cognitive andsocial-emotional control abnormalities. Impairments in baselineattention further exacerbate cognitive and social impairments andcontribute to the social anxiety and withdrawal that can lead toprofound degradation of quality of life.

Behavioral and neuroimaging studies in MDD have revealed dysfunctionalcognitive and social control systems, with marked deficits in thedorsolateral prefrontal cortex (DLPFC) and dorsal ACC in the formercase, and in the amygdala, rostral anterior cingulate cortex (rACC) andother para-limbic structures in the latter 18. Slower informationprocessing is broadly expressed across cognitive and social-controlsystems, and within the perceptual and cognitive processing machinerythat sub-serves them. Deficits in alertness, a key impairment in MDD,have been attributed to dysfunction within a broad network of regionsthat include the locus coeruleus (LC) in the brainstem as well as medialprefrontal and inferior frontal-parietal cortical areas, predominantlyin the right hemisphere. The LC synthesizes norepinephrine, anexcitatory neurotransmitter intimately involved in arousal. LC neuronswidely innervate and normally amplify responses in the forebrain, withespecially strong effects in frontal areas that are dysfunctional inMDD. Importantly, these same areas—most notably the amygdala, inferiorfrontal-parietal cortex and medial prefrontal regions—project back tothe LC, regulating its activity. Metabolically down-regulated cells inthe LC in MDD patients are reduced in sizes and numbers, and havegreatly dis-elaborated cortical terminal projections. Sizes, metabolismand terminal distributions of LC neurons are all correlated withdepression severity, suicide risk, and other life-quality variables.

In spite of a myriad of psychological and pharmacological therapies forMDD, there is still no effective treatment for a large proportion ofpatients. When treatments do help overcome depressive symptoms,underlying neurobehavioral impairments (e.g., processing speed,cognitive control processes, novelty seeking) commonly remainuncorrected. In addition, reduced cognitive control and abnormalpost-error adjustments have been described in individuals with currentand past MDD, in patients with elevated dysphoria, and inpsychiatrically healthy individuals carrying genetic variants linked toMDD risk, suggesting that these deficits represent core MDDvulnerabilities.

In addition to their limited efficacy for many patients, long-term useof antidepressant medication is expensive, and often results in unwantedside effects. Problems arising from drug withdrawal and justifiable fearof relapse promote long-term—and not infrequently, life-long—drug usage.Psychotherapy (e.g., cognitive behavioral therapy that is usuallyfocused on coping with environmental stressors, cognitive restructuringof negative thoughts, and ‘consciously elevating’ mood) is a more benigntreatment approach, but given the prevalence of low arousal states anddysphoria in MDD, compliance can be poor, treatment failure rates againapproach 50%, and relapse is common.

From a neuroscience perspective, MDD originates as an experience-drivendistortion in the processes of the ‘plastic’ human brain. Psychotherapytreatments have focused on the reduction of the psychological traumas,distresses and anxieties that (among other impacts) result in adysregulation of the systems that control baseline levels of alertnessand attention as well as cognitive control and social-emotionalprocesses. Pharmacological treatments increase the circulating levels ofmodulatory neurotransmitters that are dysregulated as a consequence ofneurological distortions in the arousal and cognitive control centers inthe forebrain attention network. Neither treatment fully addresses thecomplex, emergent neurological distortions characteristic of MDD. Evenwhen effective, standard treatments require long-term if not life-longmedication or behavioral therapy, and leave the patient with a strongrisk of illness recurrence.

Traumatic Brain Injury.

About one in five Americans incur one or more ‘mild’ or ‘moderate’traumatic brain injuries that shall bring them to a hospital emergencyroom or clinic sometime over the course of their lifetimes. About 1.7million such injuries, sine qua non with diffuse brain damage, arereported in the U.S. each year. Studies in animal models and in humanpopulations have shown that the neurological impacts of such injuriesare cumulative. For example, a head injury that results in a concussionincreases the probability that an equivalent second blow to the headwill induce another concussion; that second concussion can be induced bya substantially weaker subsequent blow; and repeated concussive injuriesgenerate progressively more severe and more enduring behavioral andneurological expressions of broadly distributed brain damage.

Some populations are at especially high risk for more-severe or repeatedhead injuries. Approximately 300,000 of the 1.6 million men and womenwho have served in the armed forces in Iraq and Afghanistan, haveincurred a TBI; more than 90% of those injuries are categorized asfalling within the ‘mild’ to ‘moderate’ part of the clinical spectrum.In the very hazardous physical environment of the Iraq/Afghanistan,about one in three of these individuals have suffered repeated TBIs.

Remarkably, up to about 80% of TBI soldiers and veterans were subjectedto blast injuries, which have a high incidence for generating diffusebrain damage. The pressure waves generated by nearby explosions cangenerate vacuolization (thousands of tiny foci of damage) and inducediffuse damage of axons in both fiber tracks and ‘gray matter’throughout the brain. The neurological consequences of such injuries canbe long enduring, and can affect almost every aspect of brain function.Many tens of thousands of Iraq/Afghanistan veterans have neurobehavioraldeficits attributable to blast injuries that can be expected to degradetheir ability to function and thrive in the military, and in theirpost-military civilian lives.

In the US civilian population, about half a million individuals incur amedically-reported concussive injury arising from sports or leisureactivities each year. While the single most common cause of a concussiveTBI in the civilian population is a bicycling accident, a more seriousmedical challenge arises from contact sports like boxing, hockey,American football, lacrosse or soccer, in which there is a highprobability of repeated brain injury. Studies using sensors mounted inthe helmets of American football players, for example, document about athousand potentially-brain-damaging blows incurred through a high-schoolor college career for a typical individual American football player. Itshould be noted that there has been a long-standing presumption thathead blows that do not result in concussion present little risk for anathlete, but many animal studies and more-current human studieschallenge this proposition. Clear evidence of physical brain damage canbe recorded in non-concussed collegiate football players through thecourse of a playing season. There are professional and collegiateplayers who acquire a head injury-induced form of early senility who hadlittle or no history of concussions during their playing careers. For aprofessional football player, cumulative brain injuries almost certainlyaccount for their nearly 20-fold increase in their risks for early-onsetAlzheimer's disease. Years of added risk portending an earlier onset ofsenility appear to result from engagement in any contact sports to thelevel of a professional or collegiate athlete.

Traumatic brain injuries commonly induce other neurological problemsthat can further degrade cognitive abilities, and the qualities of lifeof injured individuals. The majority of TBI patients have post-injurysleep disruption, a problem that can be long enduring. Most haverecurring headaches that can plague the TBI sufferer long after theirinjury. Diffuse brain injury generates abnormal, destabilizing brainactivities not infrequently expressed as epileptiform ‘sharp spikes’, orless commonly, by emergent frank epilepsy. A TBI sharply increases therisks of onset of major depressive disorder. Repeated head trauma canresult in a neurodegenerative condition called ‘chronic traumaticencephalopathy’ that foretells Alzheimers-like pathology emerging at ayoung age. As noted earlier, the occurrence of a TBI very significantlyshortens the predicted time to onset of Alzheimer's Disease itself.Finally, TBIs arising from a traumatic experience—or in individuals likemilitary veterans, law enforcement officers or health care professionalswho might be exposed to repeated traumatic events—are often accompaniedby post-traumatic stress disorder (PTSD). Given the overlap in theneurological expressions of PTSD and TBI, diffuse brain injury verysubstantially increases the probability that PTSD will arise in anindividual who has experienced, or subsequently experiences disturbingevents. Co-morbid PTSD significantly increases the TBI patient'sneurological burden and cognitive impairments, and very significantlyimpedes their passage back to a normal, stable and productive life.

A number of these problems emerge and grow after the TBI incident,indicating that damage sets destructive change processes in motion thatcan progressively amplify dysfunction. Headaches, neurologicalinstability, depression, chronic traumatic encephalopathy, PTSD andother associated sequelae can all contribute to what can be growingproblems for a traumatically brain injured individual.

Diffuse traumatic brain injuries induce immediate, widely distributeddamage to axonal connections in the brain, and to both subcortical andcortical “gray matter.” The physical blow or blast appears to result inbreakage of the stiff microtubules that transport nutrients,neurotransmitters and other materials in axons, supporting axon andterminal (synapse) vitality. As a result of this and other damage, thereis a significant diffuse loss of axonal projections and synapses. Thedisruption of axonal transmission and the local swelling anddegeneration of axons manifest thousands to millions of these“micro-damage” events in the human brains of a typical TBI-affectedpatient, with the regions of maximum damage roughly associated with thedomains of most-significant neurobehavioral losses that result from thetrauma. In a healthy, young brain, there can be substantial physicalrecovery from these losses, in the sense that axonal swelling anddieback can recover after the initial injury. However, losses inconnectivity incurred by the TBI degrade local brain connectivity andreduce connectional reliability, and greatly increase intrinsic brainprocess “noise” (neuron network “chatter”).

Although there is substantial individual variability in this expressedpathology, damage relatively predictively and disproportionately affectscertain neuronal systems and processes. For example, changes in bloodperfusion patterns, alterations in resting state connectivity, and thedocumentation of distortions in neurological responses evoked byspecific explicit behaviors known to be affected by “mild” or “moderate”TBI record the most prominent physical and functional changes in thesubcortical caudate nucleus, thalamus and cerebellar vermis, and inmiddle and lateral anterior frontal cortex, the superior temporal cortexand the posterior cingulate cortex. At the same time, a large body ofevidence has shown that the functionality and sustained connectivity ofthese specific brain areas are strongly dependent on the integrity ofthe machinery and the quality of the information at “lower levels” inthe complex neurological systems that feed them—indicating that therecovery of the physical integrity and the functionality of thesesystems represent the real therapeutic targets.

Most civilian and military TBI patients have speed-of-processingdeficits. Such deficits, grossly impacting the efficiency ofneurological operations in recognition and responding, are associatedwith that increase in “noise” (“chatter”) in the TBI brain, withweakened inhibitory processes affecting widely distributed brain areas.Again, the microtrauma-induced damage to axonal projection pathways andthe reduction of elaboration of connectivity within brain networks isthe probable primary source of this increased chatter.

A large proportion of patients have attention deficits expressed bylowered baseline levels of arousal or attention, and by impairments inselective and sustained attention. A heightened susceptibility todisruption of attention by distractors often adds to the TBI patient'sdifficulties at staying on task in attention- and memory-demandingbehaviors. Sleep regulation deficits also stem from this dysregulationof arousal, attention control and distractor control processesassociated with TBI.

Many individuals with TBIs have deficits in working memory, memory span,and delayed recall. Deficits are sometimes not evident on standardizedtesting, but are revealed when the memory task engages divided attentionor involves multi-tasking, or is evaluated in more-cognitively-demandingtask scenarios.

Working memory contributes importantly to “cognitive control” abilities;deficits in these higher-order cognitive control processes have beenrepeatedly documented in TBI. Not surprisingly, those deficits in“cognitive control” or “executive control” have been correlated indifferent studies with both processing speed and working memorydeficits. In this domain of cognitive control, individuals with TBIoften have special problems in reward discounting and in associatedimpulse control and aggression that almost certainly contribute to theirgreater risks for succumbing to substance abuse and other addictivebehaviors. These deficits are especially marked in individuals withco-morbid PTSD.

Problems in social cognition and social control can be especiallyimpactful for an individual with TBI because a degradation of socialcognition can contribute so importantly to employment success, and tothe effective reconnection of the brain-injured individual with theirpartners, families and communities. About half of individuals with TBIshave difficulties in recognizing and responding appropriately to facialaffect or gesture-expressed emotions; a larger proportion have problemsin higher-order aspects of social cognition that impact interactivesocial skills, attachment and empathy.

Childhood Abuse.

Stressed and abused children who endure multiple negative factors intheir social environments express altered levels of cortisol andnoradrenaline in their bodies and brains. While thecortisol/noradrenaline responses to stress underlie our effectivesomatic and neurological responses to danger/threat that help assure oursurvival, unabated stress (cortisol & noradrenaline release) hasenduring negative functional and physical impacts on elemental learningprocesses and on the modulatory control machinery governinglearning-induced plasticity in their brains. High circulating levels ofnoradrenaline and the delayed maturation of inhibitory processes in thebrain contribute to a greatly elevated risk of onset of an anxietysyndrome. At the same time, paradoxically, the brain's own production ofnoradrenaline, dopamine, serotonin and acetylcholine—all key“neuro-modulators”—are down-regulated, which, paradoxically, results ina weakened resilience against the later onset of a depressive disorder.

There are more than two million Americans with a history of abuse inwhich these contrary neurological effects ultimately cycle from a periodof down- to up- to down-regulation of these processes, expressed as anemergent bipolar disorder. Moreover, many stressed and abused childrenhave attention control deficits encompassing problems with both a)inattentiveness and b) ‘hyperactivity’ associated with impulsivity anddifficulty in controlling responses to distractors. These changesobviously relate to the down-regulation of intrinsic noradrenalinerelease and to blunted responses to cortisol release that stem fromtheir strong engagement of the HPA axia in periods of stress or abuse.Furthermore, most distressed and abused children have distortions inreward-weighting processes in their brain that, combined with theircognitive control deficits and impulsive responding, put them at highrisk for the later emergence of destructive addictive and compulsivebehaviors.

Also, most distressed and abused children have deficits in socialcognition that impair social interaction success and weaken theirdevelopment of attachments and empathy. These deficits, which contributestrongly to a degraded quality-of-life, also foretell a greatlyincreased probability that societal alienation shall ultimately resultin criminal offense and incarceration. They frustrate the chances that achild that has been subjected to ongoing stress or abuse shall have athriving, social, successful older life

SUMMARY

A training program is configured to systematically drive neurologicalchanges to treat MDD. The training program comprises a plurality ofcomputerized games, including an inference renormalization gameconfigured to present a plurality of stimuli, one of which is a freezestimulus belonging to a category, a first subset of which is otherfreeze stimuli that do not belong to the category, and a second subsetof which is stimuli that do not belong to the category, includingnegatively affective stimuli. The game participant is prompted torespond to only the first set of stimuli and refrain from responding tothe freeze stimulus and distractor stimuli.

In another embodiment, the inference renormalization game is configuredto present a first subset of positively affective stimuli, a secondsubset of negatively affective stimuli, and a third subset of neutralstimuli. The game participant is prompted to respond according to afirst mode when presented with positively or negatively affectivestimuli, and according to a second mode when presented with neutralstimuli. One of the first and second modes comprises responding througha game piece to a stimulus. The other of the modes comprises refrainingfrom responding through the game piece to the stimulus;

Other features and advantages of the present invention will becomeapparent upon study of the remaining portions of the specification anddrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of one embodiment of a multi-faceted,web-deliverable, and game-based neurological training system configuredto treat one or more mental disorders or cognitive deficits.

FIG. 2 is a block diagram of one embodiment of a curriculum of aneurological training program configured to treat one or more mentaldisorders or cognitive deficits.

FIG. 3 is a block diagram of an embodiment of a hierarchicalorganization of games within the neurological training program.

FIG. 4 is a block diagram of one embodiment of a computer-basedcognitive training program that is configured to treat major depressiveorder.

FIG. 5 is a block diagram of one embodiment of a computer-basedcognitive training program that is configured to treat traumatic braininjury.

FIG. 6 is a block diagram of one embodiment of a computer-basedcognitive training program that is configured to treat child abuse.

FIG. 7 illustrates a screenshot of one embodiment of a divided attentiongame called “Task Switcher,” which challenges a game participant toidentify whether two objects share a target characteristic, wherein thetarget characteristics change unpredictably.

FIG. 8 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 9 illustrates a screenshot of one embodiment of a game called“Wango Mango,” which challenges a game participant to press the spacebarwhenever an image appears that is not the target, but to withholdpressing the spacebar if the image appears.

FIG. 10 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 11 illustrates a screenshot of one embodiment of a working memorygame called “Spatial Memory,” which challenges a game participant tomemorize the locations of several objects and thereafter respond if theobject is at a remembered location.

FIG. 12 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 13 illustrates a screenshot of one embodiment of a navigation gamecalled “Brainville,” which challenges a game participant to recall theprogression of a journey, the objects seen along that journey, and theorder in which those objects were seen.

FIG. 14 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 15 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 16 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 17 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 18 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 19 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 20 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 21 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 22 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 23 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 24 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 25 illustrates a screenshot of one embodiment of a visuospatialmemory game called “Visual Memory,” which challenges a game participantto remember objects shown at different spatial locations across a timedelay.

FIG. 26 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 27 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 28 illustrates a screenshot of one embodiment of an auditory andspatial memory game called “Look, Listen, and Repeat,” which challengesa game participant to press buttons he sees and hears in the rememberedorder.

FIG. 29 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 30 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 31 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 32 illustrates a screenshot of one embodiment of a sequencing andmultimodal integration game called “Keyboard: Timing,” which challengesa game participant to recall the rhythm of a musical sequence by playingit back on the spacebar.

FIG. 33 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 34 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 35 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 36 illustrates a screenshot of one embodiment of a sequencing andmultimodal integration game called “Voice: Timing,” which challenges agame participant to recall the rhythm of a musical sequence by singingthe melody back.

FIG. 37 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 38 illustrates a screenshot of one embodiment of a sequencing andmultimodal integration game called “Voice: Timing & Pitch,” whichchallenges a game participant to recall the rhythm and pitch of amusical sequence by singing the melody back.

FIG. 39 illustrates a screenshot of one embodiment of an executivefunctioning game called “Rule Change,” which challenges a gameparticipant to categorize cards based on conditional rules.

FIG. 40 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 41 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 42 illustrates a screenshot of one embodiment of another executivefunctioning game called “Mental Rotation,” which challenges a gameparticipant to mentally rotate a letter or number to its right-side upposition and indicate whether it is a normal or mirrored image.

FIG. 43 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 44 illustrates a screenshot of one embodiment of a mindfulnessexercise called “Air Control,” which challenges a participant topractice mindful breathing.

FIG. 45 illustrates another screenshot of the exercise illustrated inthe previous figure.

FIGS. 46-65 illustrate various embodiments of attentional biasmodification games that challenge the game participant to suppressresponses to stimuli to which the game participant is prone toward anunhealthy psychological response, such as craving, trauma, ordepression.

FIG. 46 illustrates a screenshot of one embodiment of a game called“Grin Hunting,” which challenges a game participant to ignore stimulithat trigger unhealthy psychological responses and/or selectivelyrespond to healthy stimuli.

FIG. 47 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 48 illustrates a screenshot of one embodiment of a game called“Category Click,” which challenges a game participant to selectivelyrespond to stimuli that fall within a target category except for one ormore pre-identified “freeze” stimuli that fall within that category.

FIG. 49 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 50 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 51 illustrates a screenshot of one embodiment of a game called“Mood Matchmaker,” which challenges a game participant to match achallenge stimulus to a target stimulus in a context that includes aplurality of distracting stimuli, including stimuli that triggerunhealthy psychological responses.

FIG. 52 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 53 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 54 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 55 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 56 illustrates a screenshot of one embodiment of a game called“Name the Color,” which challenges a game participant to respond toidentify the font color of words that include words that triggerunhealthy psychological responses.

FIG. 57 illustrates a screenshot of one embodiment of a delaydiscounting game called “Now or Later,” which scores a gameparticipant's performance based upon the degree of self-control,including willingness to accept delayed gratification, that the gameparticipant exhibits in selecting choices.

FIG. 58 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 59 illustrates a screenshot of one embodiment of a game called“Scene That!” which challenges a game participant to indicate whether atarget stimulus was contained within a set of distracting stimuli,including one or more stimuli that trigger unhealthy psychologicalresponses.

FIG. 60 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 61 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 62 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 63 illustrates a screenshot of one embodiment of a game called“Tick Tock,” which displays an object and challenges the gameparticipant to select the object after a user-perceived time interval.

FIG. 64 is a block diagram illustrating multiple embodiments of TickTock.

FIG. 65 illustrates a screenshot of one embodiment of a working memorygame, called “The Matrix Recalled,” that presents an ordered list ofitems—which occasionally includes an item that stimulates an unhealthypsychological response—and challenges the game participant to recall theitems in the requested order.

FIGS. 66-143 illustrate various embodiments of social cognition gamesthat challenge game participant with social impairments to recognize,understand, and respond to social, visual, and auditory cues.

FIG. 66 illustrates a screenshot of one embodiment of a facial cueprocessing speed game called “Gaze Cast,” which presents a video clip ofa person making a speeded glance shift in one of many possibledirections followed by an array of peripheral objects and challenges theparticipant to select the peripheral object in the direction of theperson's glance.

FIG. 67 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 68 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 69 illustrates a screenshot of another embodiment of a facial cueprocessing speed game called “Looky Lou,” which presents a target facefor a brief interval of time, followed by a visual mask, followed by asubsequent set of faces, and challenges the participant to select theface whose eyes are gazing in the same direction as the target face.

FIG. 70 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 71 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 72 illustrates a screenshot of yet another embodiment of a facialcue processing speed game called “Face It,” which presents a target facefrom the front, side, or an angle, followed by a visual mask, followedby a subsequent set of faces, and challenges the participant to identifythe target face in the array.

FIG. 73 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 74 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 75 illustrates a screenshot of one embodiment of an emotional cueprocessing speed game called “Name That Feeling,” which presents atarget face expressing an emotion, followed by a visual mask, andchallenges the participant to indicate the emotion that best expressesthe emotion exhibited by the target face.

FIG. 76 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 77 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 78 illustrates a screenshot of one embodiment of an emotional cueprocessing speed game called “Emotion Motion,” which presents a videoclip of a face expressing an emotion, followed by a visual mask, andchallenges the participant to indicate the emotion that best expressesthe emotion exhibited by the target face.

FIG. 79 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 80 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 81 illustrates a screenshot of yet another embodiment of anemotional cue processing speed game called “Poke That Feeling,” whichpresents a target image of a face expressing an emotion, followed by avisual mask, followed by a set of facial images each expressing adifferent emotion, and challenges the participant to select a facialimage whose expressed emotion best matches the emotion expressed by thetarget image.

FIG. 82 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 83 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 84 illustrates a screenshot of one embodiment of a processing speedgame called “Mass Affect,” which presents a target image that has acharacteristic valence and challenges the participant to match it withanother image having a similar valence.

FIG. 85 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 86 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 87 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 88 illustrates a screenshot of one embodiment of a processing speedgame called “TAPAT MDD,” which presents a series of smiling, frowning,and neutral facial images and challenges the participant to press thespacebar when seeing a smiling or frowning image but to withholdpressing the spacebar when seeing a neutral image.

FIG. 89 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 90 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 91 illustrates a screenshot of one embodiment of a game called“Bright Whites,” which repeatedly challenges the participant to adjustto changing emotional outputs by two characters and identify whichcharacter was most recently smiling.

FIG. 92 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 93 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 94 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 95 illustrates a screenshot of one embodiment of a game called“What Just Happened?,” which plays short video clips of actorsexpressing emotion or a neutral face followed by challenging theparticipant to identify which of multiple options would best describe anexperience that elicited the actors' expressed emotions.

FIG. 96 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 97 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 98 illustrates a screenshot of one embodiment of a working memoryemotional cue game called “Second That Emotion,” which presents an arrayof down-facing cards and challenges a participant to select card pairsthat show matching facial emotions.

FIG. 99 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 100 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 101 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 102 illustrates a screenshot of another embodiment of a workingmemory prosody game called “Second That Intonation,” which presents anarray of down-facing cards associated with spoken sentences andchallenges a participant to select card pairs that match a spokensentence's prosody with a label.

FIG. 103 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 104 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 105 illustrates a screenshot of one embodiment of a memory spa gamecalled “Face It: Flashback,” which presents a sequence of faces followedby challenging the participant to select the faces in the order theywere presented.

FIG. 106 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 107 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 108 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 109 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 110 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 111 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 112 illustrates a screenshot of one embodiment of a social memoryspan game called “Face Facts,” which presents pictures of individualstogether with facts about each individual and challenges the participantto select facts that are true about the individuals.

FIG. 113 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 114 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 115 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 116 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 117 illustrates a screenshot of one embodiment of a prosodyapprehension game called “Voice Choice,” which challenges the gameparticipant to identify the emotion of a neutral sentence spoken with anemotional prosody.

FIG. 118 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 119 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 120 illustrates a screenshot of one embodiment of a social cueapprehension game called “Life Stories,” which challenges gameparticipants to listen to a story and apprehend social details in thestory.

FIG. 121 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 122 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 123 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 124 illustrates a screenshot of one embodiment of a vocal affecttheory of mind game called “Say What?,” which challenges gameparticipants to apprehend a social situation and the meanings conveyedby voice inflection.

FIG. 125 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 126 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 127 illustrates a screenshot of one embodiment of a namememorization game called “Face and Name,” which challenges gameparticipants to associate a plurality of names with a plurality offaces.

FIG. 128 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 129 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 130 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 131 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 132 illustrates a screenshot of one embodiment of a vocal emotionalcue and theory of mind game called “Auditory Chatter,” which challengesgame participants to answer questions about persons discussed in asocial conversation.

FIG. 133 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 134 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 135 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 136 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 137 illustrates a screenshot of one embodiment of a theory of mindgame called “Social Scenes,” which presents a short written narrativeabout a person and challenges the participant to infer what that personwould have thought or felt based on the narrated circumstances.

FIG. 138 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 139 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 140 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 141 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 142 illustrates a screenshot of one embodiment of a vocal cuetheory of mind game called “What's Joe Thinking?” which challenges gameparticipants to follow the direction of different people's eye gazes andinterpret those gazes as denoting their thoughts.

FIG. 143 illustrates another screenshot of the game illustrated in theprevious figure.

FIG. 144 illustrates another screenshot of the game illustrated in theprevious figure.

DETAILED DESCRIPTION

Various embodiments of the present invention use a computer system and acomputer network for executing one or more computer programs to train orretrain an individual to enhance cognition, where the term “cognition”refers to the speed, accuracy and reliability of processing ofinformation, including filtering, recall, and manipulation ofinformation, and attention and/or working memory.

A typical computer system (not shown) for use with the present inventionwill contain a computer, having a CPU, memory, hard disk, and variousinput and output devices. A display device, such as a monitor or digitaldisplay, provides visual prompting and feedback to the subject duringexecution of the computer program. Speakers or a pair of headphones orear buds provide auditory prompting and feedback to the subject. Aprinter may be connected to the computer to enable a subject can printout reports associated with the computer program of the presentinvention. Input devices such as a keyboard, mouse, trackpad, touchscreen, microphone, camera, or other sensor receive input from thesubject. Although a number of different computer platforms areapplicable to the present invention, embodiments of the presentinvention execute on either IBM compatible computers or Macintoshcomputers, or similarly configured computing devices such as set topboxes, PDA's, gaming consoles, etc.

A computer network (not shown) for use with the present inventioncontains multiple computers similar to that described above connected toa server. The connection between the computers and the server can bemade via a local area network (LAN), a wide area network (WAN), or viamodem connections, directly or through the Internet. The computernetwork allows information such as test scores, game statistics, andother subject information to flow from a subject's computer to a server.An administrator can review the information and can then downloadconfiguration and control information pertaining to a particularsubject, back to the subject's computer.

I. GENERAL CHARACTERISTICS

FIG. 1 is a block diagram of one embodiment of a multi-faceted,web-deliverable, browser-playable and game-based neurological trainingsystem 1 configured to treat a cognitive deficit. The neurologicaltraining system 1 comprises a game platform 12 and training program 2.The game platform 12 controls exercise delivery and records all data ongame play and performance progressions for exercise suites playable onInternet-connected computers and pads. It comprises a program manager 3,participant portal 8, clinician portal 10, and database 9. The programmanager 3—also referred to herein as a game manager—is configured toadminister the training program 2, manipulate a plurality of gamestimuli 5, and receive input from at least one game piece 4.

The training program 2 comprises a plurality of games or exercises 4targeting a plurality of cognitive domains and sensory modalities. Eachtraining program 2 is customized and configured to address cognitivedeficits that are associated with a neurological condition, such asaddiction, depression, ADHD, or ASD, and its co-morbidities.

Each training program 2 exercises a range of cognitive domains, fromfoundational ones like processing speed to more complex ones like socialcognition. Ordered from most foundational to most complex, the cognitivedomains targeted comprise processing speed 26, attention 27, workingmemory 28, executive functioning 29, self-control 30, and socialcognition 31.

In most embodiments, the game stimuli comprise images 23 displayed on adisplay device such as a computer monitor or digital screen and/orsounds 24 played through a speaker, ear buds or other auditoryequipment. In other embodiments, the game stimuli comprise smells,tastes, or tactile (e.g., haptic) stimulation 25. The training program'sstimulus set is designed to span the relevant dimensions of real-worldstimuli to ensure that learning is never stimulus specific.

Early in training, the games use highly salient, emphasized (e.g., highcontrast, temporally deliberate) stimuli 21 to drive stronglysynchronized brain responses requisite for rapidly driving brain changesin a corrective way. The games then progressively move to moreecologically-relevant and valid stimuli 22 (e.g., real speech, complexrealistic social stimuli with people showing emotions in context, socialscenes, social interactions) to ensure generalization to real-worldsituations. However, in games to bias attention away from an unhealthystimulus, the games progress the opposite direction, from low-salience22 to high-salience stimuli 21.

The game piece 6 comprises a keyboard, computer mouse, track pad, touchscreen, camera, remote sensing device (e.g., Microsoft Kinect®),microphone, or other input device.

The training program 2 provides the games through a portal 8 that isdesigned to be played in a social network environment, at a treatmentcenter, or during a treatment class. In one embodiment, the trainingprogram 2 is designed to be platform-independent so that it can bedelivered over the Web via any Internet-connected computer. In anotherembodiment, the training program 2 is provided through a hand-heldcomputer (iPhone/Android phone/iPad/Android tablet/Amazon Fire)application.

The participant portal 8 provides access to game participants.Practically any patient on any computer located anywhere in the worldcan work on these programs as frequently as their time and schedulepermit, under the supervision of a clinician who can (hypothetically)also be located anywhere. To use the program, a participant opens astandard web browser on a broadband connected computer and goes to aprogram web site. The participant then logs into the program using ascreen name that contains no personally identifiable information.

In one embodiment, the portal 8 introduces the participant to a“meta-game wrapper” such as an image and map of a virtual social citythat allows participants to visit locations, access games, view progressand results, and make or communicate with friends. The meta-game wrapperis characterized by simplicity, appealing graphics, a sense of control,and constant game rewards.

The program manager 7 is configured to administer the games 4 to a gameparticipant in a roughly hierarchical—but not purely linear—fashion. Asillustrated in FIG. 3, each game comprises a plurality of blocks 16 oftrials 15. Each game trial 15 presents a stimulus or stimuli 20,challenges the game participant to respond to the stimuli 20, receivesthe game participant's input through the game piece 4, and provides anindication or measure of the game participant's performance, accuracy,and/or aptness.

The program manager 7 applies a delivery strategy that subdivides eachgame 4 or training module into progressive 1.5-2 minute progressivetraining challenges (or trials 15). Initial task levels are achievableby every participant, but every participant is driven forward inperformance to an asymptotic limit at the end of every short trainingcycle. The participant is then asked to repeat the cycle to beat theirjust-achieved score. Achievement is further marked by a “star” system inwhich five stars (the maximum) represents normal ability for aparticipant of the trainee's age at that specific task. In this exerciseformat, the participant is encouraged to replay subtasks in which theirperformance is still improvable, to get more and more “stars.”Participants also work for rewards and achieving goals in an over-ridingmeta-game that has a structured reward economy.

Between blocks 16 of trials 15, the program manager adapts one or moreparameters—such as the salience, duration, and number of target stimuli20, the salience, duration, and number of distractor stimuli, and thetime period between which the target stimuli are presented and theprogram manager 7 prompts the participant to answer questions thatrelate to the target stimuli—affecting a difficulty of the game 4 inresponse to the game participant's input.

Typically, games also include multiple levels 17 and multiple stages 18,each level 17 having a plurality of blocks 16 and each stage 18 having aplurality of levels 17. In a typical multi-level game 4, the gameparticipant is required to achieve a certain threshold of performance inorder to “unlock” another, more difficult level 17. Many of the gamesare also grouped into progressive (and optionally overlapping) tiers 20,requiring the game participant to satisfactorily complete all of thegames 4 in a given tier 20, and a sufficient number of levels 17 withinthose games 4, before providing access to the games 4 of the next tier20. Games 4 are grouped so that each progressive tier 20 includes gamesthat span a variety of neurological target categories 32.

The program manager 7 administers a schedule that ensures that aparticipant progress through the games 4 in a defined order, generallymoving from more simple (early sensory processing) games 4 to morecomplex (multimodal, working memory, memory span) games 4 over thecourse of a multi-week experience. At any point in time, the participanthas access to a subset (for example, eight) of these games 4, and ischallenged to perform at least a certain number (for example, six) ofthe games 4 per day. Each game 4 has specific criteria for completion orplateaued performance. After those criteria are met, the game 4 isremoved from the active set and the next game 4 added to it. Thismechanism ensures ongoing novelty and engagement for the participant,while ensuring that the participant progresses smoothly through thecomplete set over the program use period.

Within each game 4, a performance threshold, determined by anappropriate up-down procedure or a Zest algorithm, is derived for eachblock completed. The performance threshold provides performance andimprovement data on the individual games. Game-based assessments, whichare designated blocks 16 with medium difficulty of the specific games 4,are performed at various time points in the intervention to checkprogress.

The games 4 in the training program 2 are designed with a differentobjective than conventional games. Conventional games start at a fixedbaseline and progress in a single direction, getting more difficultuntil the participant is unable to go any further, at which point thegame typically terminates. Conventional multilevel games also requirecompletion of one level to progress to the next, more difficult, level,terminating mid-level if the participant is unable to complete a givenlevel.

The games 4 of the training program 2, by contrast, are adaptive andthreshold-based. Learning rules are relaxed in initial training toassure near-errorless learning. Error rates are slowly increased toachieve challenge conditions known to be optimal for normal individualsby the end of the training epoch. Likewise, ‘rewards’ for performancegains are initially amplified, in comparison with those applied fortraining in other patient populations. The games 4 increase indifficulty when the participant exceeds a threshold of success, and theydecrease in difficulty when the participant's performance drops belowanother threshold. Many of the games 4 enable a participant to “unlock”a new level merely by beating the participant's previous best score. Bymeasuring success in metrics of personal improvement rather than fixedperformance requirements, both participants with relatively highcognitive abilities and participants with relatively significantcognitive deficits can progress through the entire training program 2.

After logging in, a game-like experience begins in which the participantis encouraged to earn points and in-game rewards to further advance ineach game 4. To do so, the participant selects one of the games 4scheduled for the day, and performs that game for 5-10 minutes. The game4 itself contains the core science stimuli and task built into agame-like experience. Performing the game 4 resembles practice on askill akin to learning to play an instrument or learning to ride abicycle.

Participants perform tens to hundreds of trials 15 over the course ofthe ten-minute session. Each trial 15 provides auditory and visualfeedback and rewards to indicate if the trial 15 was performed correctlyor incorrectly. After each trial 15, the difficulty of the next trial 15is updated to ensure that within each session, the participant gets ˜85%of trials correct. Maintaining a relatively high level of success helpsprevent frustration and minimizes the possibility of potential drop-outfrom the program. Summary screens including game metrics (points,levels) and game metrics (usage, progress) are shown to the participantat the end of each session.

To progress through a game 4, the participant performs increasinglydifficult discrimination, recognition, memorization or sequencing tasksunder conditions of assured focused attention. Each game 4 employsadaptive tracking methods to continuously adjust one or two adaptivedimensions of the task to the sensory and cognitive capabilities of theparticipant. This process is based on a statistically optimal Bayesianapproach that allows the games 4 to rapidly adjust to an individual'sperformance level, and maintain the difficulty of the stimulus sets atan optimal level for driving most-efficient learning.

This continuously-adjusted adaptivity operates from trial 15 to trial15, to sustain an individual's performance success at a challenging(80-90%), since subject is not correct all the time, yet engaging andrewarding (since subject is correct most of the time) level ofperformance success. This continuously-adjusted adaptivity is alsoadjusted across sessions to ensure that the games 4 become morechallenging at exactly the appropriate rate for a specific individual'srate of learning. This adaptivity also allows the game 4 to adapt to anindividual's variable performance across days depending on their overallmood, attention, and health.

By this strategy, training is individualized. A trainee rapidlyprogresses across training landscapes for which impairments are mild orinsignificant but must work hard to improve domains of substantialimpairment—always working on the edge of their achievable performanceabilities to drive positive, corrective changes at an optimizedday-by-day rate, to address the specific neurological problems that mostspecifically frustrate their higher functioning.

If a game 4 is used as a training module, it is presented as stages 18that last about ten minutes. During those ten minutes, the participantplays the stage 18 two times: first to set a baseline, and second tobeat or match that baseline. This repetition is intentional, because thetargeted strengthen of certain neural pathways achieved in the gamesrequires repeated activation of those neural pathways. Stages 18generally have one level 17 (block of trials intended to be playedstraight through), but they can have more.

The program manager 7 delivers all patient performance data in encryptedforms to a cloud-located database, which are provided, with appropriateinformed consents, to one or more treatment program professionals, whoaccess the relevant patient data through a clinician portal 10 in orderto supervise patient treatment and assure enrollment, compliance, andmonitored and guided patient progress.

Every aspect of a patient's compliance and progress is recorded intraining and can be provided via a cloud-based database 9 (withappropriate permissions) to supervising training monitors orprofessionals. No personally identifiable information (includingInternet protocol addresses) is stored on the server. The server makesthe data available for review by the clinician(s) supervising patientcare, the site trainer, site coordinator, and site investigator througha secure web portal 10, which requires a complex password to secure theidentification of these individuals. Only data from participants in aparticular clinic can be viewed by that clinic's staff. The sitetrainer, in particular, uses the secure web portal 10 to regularly checkon usage and progress of each active participant to customize theirweekly phone/in-person/social network discussions to provide helpfulguidance and coaching.

By solving the treatment access problem and by providing a basis forintensive and extensive remotely-supervised treatment at low cost,substantially improved overall medical outcomes in this population canbe achieved.

II. ASSESSMENTS

Each training program 2 utilizes assessments to personalize the types ofgames, types of stimuli, and levels of difficulty to the participant.Each game 4 can be used as an assessment or a training module. If a game4 is used as an assessment, it is played once through before theparticipant advances to the next game.

Playing an assessment typically lasts five or fewer minutes. Assessmentsare used sparingly to avoid inducing learning/practice effects. Inassessments, progressive variables that change an exercise's difficultymay be removed to provide a static reading of performance. Visual and/orauditory feedback and rewards may also be removed to reduce effectsrelated to trial-by-trial learning or motivation.

The training program 2 uses two general forms of assessments tocalibrate the games 4, providing objective measures like reaction timesand accuracy or subjective measures like self-reported abilities.Assessments provide a range of information regarding within exerciseimprovements (is the participant improving on the task at hand?),near-transfer effects (is training on a processing speed task going tolead to improvements on other processing speed tasks?), far-transfereffects (is training on an attention task going to lead to improvementson a memory task?), and clinical and real world measures (is ourtraining program going to improve depressive symptoms or quality oflife?)

The training program 2 also uses standard clinical andneuropsychological assessments. The training program 2 uses the TrailMaking Test and Wechsler Adult Intelligence Scale (WAIS-IV) SymbolCoding to assess processing speed. The training program 2 uses theWAIS-IV Digit Span and Arithmetic to assess working memory, the HopkinsVerbal Learning Test to assess delayed memory, and the BriefVisuospatial Memory Test to assess visuospatial memory. The trainingprogram 2 uses the Trail Making Test and Wisconsin Card Sorting Test toassess executive functioning. The training program 2 uses the IowaGambling Task, Balloon Analog Risk Task (BART), and BarrattImpulsiveness Scale (BIS) to assess impulse control. An embodiment ofthe training program 2 used for treating alcoholism uses the PennAlcohol Craving Scale (PACS) to quantify alcohol cravings in alcoholicparticipants and Hopkins Verbal Learning Test-Revised (HVLT-R) toquantify verbal learning and memory, respectively. These assessments areadministered to participants before they begin training.

In training programs for children, surveys are provided to thechildren's guardians before and after training to assess behaviorrelated to ADHD, social abilities, conduct, and other dimensions ofdevelopment.

Assessments are embedded in the training program 2 in form of surveysand exercises of varying similarity to those selected for training.Assessments are placed before and after training and often duringcheckpoints within training.

The training program 2 calibrates the games 4 based on pre-trainingassessment results to select games 4 and certain stages or levels withinthe games to present as part of the training program. Theassessment-based calibration also includes adjusting the proportion oftraining exercises related to each cognitive domain (e.g., processingspeed, attention, theory of mind, impulse control). Calibration is alsoused to adjust parameters within an exercise like the start point of theprogressive variable or the number of trials in a block. Such parameterstend to be calibrated for adults with moderately high cognitiveabilities. Young children and individuals suffering from cognitiveimpairments often require specialized settings.

Checkpoints generally occur every 10 to 14 days of training and can bespaced further apart for longer training programs.

III. SPECIFIC TRAINING PROGRAMS

1. Depression

FIG. 4 is a block diagram of one embodiment of a computer-basedcognitive training program 60 that is configured to treat majordepressive order and mood and anxiety disorders. The primary emphasis ofthis training program 60 is on restoring and strengthening dysfunctionalaspects of attention and executive control.

The treatment program 60 includes continuous performance task (CPT)games 62 that up-regulate baseline levels of attention and alertness,improving the functional status of the alertness network (e.g., LC-NAaxis), and further increase levels of noradrenaline release. These games62 improve the functional status of the locus coeruleus, sharply andenduringly increasing its amplification of cortical activities. Thisaspect, in turn, improves the “brightness” in mood and affect ofparticipants. These strong neurological “brightening” impacts have nowbeen documented in brain-injured patients (where it contributes to there-normalization of active vision in the neglected hemifield inhemispatial neglect syndrome patients), and in normal young and olderadults.

The treatment program 60 also includes games 63 designed to increase thespeed, accuracy and power of neuronal representations of perceptualinputs in auditory/aural speech and visual domains (e.g., distinguishingdifferences in exaggerated phonemic or visual stimuli). The auditoryspeed of processing and accuracy of representation and neuronal power ofparticipants are generally down-regulated in MDD and contribute to dailylife challenges. By systematically targeting these deficiencies, theprogram 60 re-normalizes processing speed in impaired brains andamplifies and renormalizes the functional status of the rACC and DLPFC.

The treatment program 60 also includes games 64 to up-regulate visualspeed and novelty-seeking to improve participant's performance in dailylife and outlook. These games 64 require the discrimination ofprogressively faster events at fixation combined with the accuratereception of brief or fast-changing events across a progressivelyexpanding visual periphery. This training demonstrably recoversmore-active vision and novelty seeking. It also sharply up-regulatesactivities in attention networks in the brain that are specificallydisconnected in depressed patients. Also, by extending and strengtheningthe participant's ‘Useful Field of View’ (UFOV), the program 60increases the participant's brain speed.

The treatment program 60 also includes a variety of executive task games65 designed to improve cognitive control and interference resolutionwhile improving cortical efficiency in fronto-cingulate pathwayscritically implicated in MDD.

The treatment program 60 also includes a variety of self-control games66 and social-cognition games 67 to improve cortical efficiency infronto-cingulate pathways critically implicated in MDD and MA, tore-balance social control, interference resolution and social-emotionalcontrol (e.g., flanker, emotional stroop tasks), and to assistparticipants in social interaction and coping. Included in this aspectare tasks such as facial affect recognition training, which broadlyimproves emotional self-monitoring abilities in stable, chronicschizophrenics. In the schizophrenia population, training effectivelyre-balances distorted responses to emotionally negative or disturbingstimuli, and re-normalizes distorted cognitive brain system patterns ofresponse in face recognition, emotion recognition and reward-assessmentbehavioral tasks. Collectively, this approach contributes to a morecomplete, more effective and more enduring brain changes to normalizethe brain function.

Table 1 below shows a suite of six games targeted to the MDD population.

TABLE 1 Suite of games incorporated into one embodiment of a MajorDepressive Disorder treatment training program Domain Game Name ModalityTask Processing Sounds Auditory Listen two sweep sounds. The SpeedSweeps sound can sweep up from a lower to higher or down from the higherto lower. Indicate the direction of the two sweep sounds in sequenceDouble Visual Identify foveal and peripheral Decision stimuli AttentionTAPAT Visual Remember the target image and withhold a response for thetarget and commit a response for distractors Grin Visual Shift attentionand determine Hunting if target image contains smile Executive DividedVisual Indicate if two figures on the Functioning Attention screen havethe same color or not as fast as possible Mixed Visual Respond to shapesbased on Signals changing rules

The training program 60 provides participants continuous feedback abouttreatment progress, expressed directly and symbolically in an engagingmeta-game 68. Frequent reward screens are presented, with game-likerewards marking the achievement of training benchmarks. Social networkcapabilities are mounted in the training program model. Participants areencouraged (with their clinician's Internet-delivered help) to formtraining “teams” in which all participants can work together on a“mission.” The training program 60 also includes a set of assessmenttools 61 that provide a basis for evaluating key default-systemcomponents in any participant. Based upon those assessments, thetraining program 60 provides a customized set of games to drivecorrective improvements.

2. Traumatic Brain Injury

FIG. 5 is a block diagram of one embodiment of a computer-basedcognitive training program 70 configured to treat traumatic braininjury. The training program 70 comprises eighteen games 4 designed toimprove neurological functionality in five key domains of loss, i.e.,processing speed/accuracy 72, arousal 73, attention 74, memory 75,cognitive control 76 and social control 77. Many of these games 4 aredescribed later in this specification.

In one embodiment, the cognitive training program 70 comprises a“mission” based meta-game wrapper 71. The training program 70 providesparticipants continuous feedback about treatment progress, expresseddirectly and symbolically in the meta-game. Frequent reward screens arepresented, with “badges” and other game-like rewards marking theachievement of training benchmarks. Social network capabilities aremounted in the training program model. Military veterans are encouraged(with their clinician's Internet-delivered help) to form training“teams” in which all participants can work together on their “mission torecovery”—helping other team members if they need it—on that path. Thesesocial network capabilities also provide a basis for veterans toreconnect with the former members of their military units or with othersin their community or region who might have had a similar militaryexperience, potentially to encourage others in possible need to seekthis form of help from appropriate professionals.

A preliminary controlled trial was conducted on twenty patients withTBI's using one embodiment of the training program 70. Like many of theother training strategies described herein, the TBI strategy wasdesigned to up-regulate baseline levels of attention, processing speed,memory and cognitive control and social control.

The purpose of the trial was to determine whether or not training gainscould be achieved in patients with blast and other war-related headtrauma. The training required about forty hours of intensivecomputer-based exercise, working on a schedule of about one hour oftraining per day.

TBI patients willingly and enthusiastically complied with the rigorousdemands of the training. All patients showed significant improvements ona cognitive battery assessing improvements in five key domains (brainspeed, attention control, speech reception and memory, visual receptionand memory, cognitive control). Even given this small sample and limitedtraining repertoire, significant, enduring improvements were recorded inall trained operational domains. These behavioral gains and paralleledby clear quality-of-life improvements.

Preliminary data from a second controlled study involving two additionalattention- and working memory-targeted training strategies for acivilian TBI patient population are showing large positive effects. Itappears that all aspects of the deficits in attention control can beremediated, and memory span and sustained memory epochs substantiallyexpanded in individuals with TBIs, using these important additionalforms of training.

There is a strong overlap in the behavioral deficits recorded inveterans with TBIs or PTSDs, or in individuals with both diagnoses. Ingeneral, co-morbid patients have more severe deficits in most of theareas described above, with differentially strong impairments (reTBI-only patients) expressed in orbitofrontal and inferior frontalcortical processes controlling reward discounting and related socialcontrol abilities, and in post-cingulate cortical and hippocampalprocesses that contribute to complex mental reconstructions in thedomains of place and time. Key aspects of these special PTSD-relateddomains of loss are, again, direct targets of the training program suitethat BPI scientists are constructing for initial DOD-supported outcomestrials.

These training strategies should greatly increase an individualtrainee's resilience against the onset of clinical depression.Similarly, the expected broad-scale re-establishment of neurologicalcontrol in perceptual, cognitive, cognitive control and social controlabilities should contribute to neurological stabilization that shouldconfer substantial resilience against the cortical instability expressedby epileptiform sharp spikes or seizures.

3. Childhood Abuse

FIG. 6 is a block diagram of one embodiment of a computer-basedcognitive training program 80 that addresses the complex array ofneurobehavioral deficits that apply for a child with a history of severestress and abuse. The training program 80 comprises forty trainingmodules that include assessments 81, continuous performance games 82,processing speed games 83, attention games 84 (some of which involvedistractor suppression), working memory games 85, executive functioninggames 86, and impulse control games 87 (involving attentional controland goal tracking), and social cognition games 88, all packaged within ahighly engaging meta-game environment 89.

The training program 80 is designed to provide seventy to eighty hoursof training, delivered for ½ hour to one hour for three to four dayseach week. The training program 80 is suitable for after-schooladministration to students.

Because an over-arching goal is to recover social cognition and socialcontrol abilities in these children, the game scenario is pro-social(designed to support human understanding and empathy) and specificallydesigned to increase the child's social interaction skills, and tostrengthen the neurological machinery that facilitates attachment andempathy.

A “clinician portal” 10 establishes a communication link betweenclinician and trainee and engages the parent/guardian in a positive,constructive way. The clinician portal 10 assures that a child or adultcomplies with and makes progress at daily exercises. It also helpsassure that conditions of probationary sentencing are carried out by anindividual under the jurisdiction of a court who is required to completetraining as a condition of probation.

4. Distorted Time Tracking

Distorted time perception is seen in various neurological andpsychiatric disorders, including addiction, attention deficithyperactivity disorder (ADHD), autism, schizophrenia, traumatic braininjury, and Parkinson's disease. It is also seen in normal aging elderlyadults. People who tend to perceive time passing more quickly also tendto discount the future more heavily, preferring small rewards deliveredsooner over bigger rewards delivered later. Abnormal time perceptioninvolves a range of brain areas—basal ganglia, prefrontal cortex,inferior parietal cortex, premotor cortex, pre-supplementary motor area,supplementary motor area, and cerebellum. Further, many of these areasoperate in networks.

Damage to the prefrontal-inferior parietal network in the righthemisphere, but not the left, results in temporal distortions. Patientswith lesions to the right hemispherical network also have difficultyswitching their nonspatial attention, though patients with lesions ineither hemisphere had attention deficits overall. Attention and workingmemory contribute to time perception even in healthy subjects.

Focusing on the timing of visual stimuli increases metabolic activity inthe corticostriatal network involving basal ganglia, pre-supplementarymotor areas, among other areas. Focusing on the color of the stimuliincreases activity in area V4. Hence, we hypothesize that attending andtraining on temporal properties of stimuli should consequently exercisethese networked brain areas by inducing plastic changes to them.

Subjective duration of stimuli is influenced by information coming fromdifferent sensory systems. An auditory event can appear shorter orlonger when presented simultaneously with conflicting visualinformation. Thus, we hypothesize that cognitive training of time maybenefit from first training single sensory modalities, and then trainingacross modalities.

IV. PROCESSING SPEED GAMES

Each training program 2 includes, and begins with, a set of processingspeed games. These games train the participant to pick up on details orchanges in visual and auditory stimuli as quickly as possible. Forexample, in a sensorimotor reaction time game (not shown) called“Spotted,” the participant is challenged to press the spacebar as soonas they see a green circle appear on the screen.

V. ATTENTION GAMES

Each training program 502 also includes a set of attention and sensorydiscrimination games. These games train the participant to focus oncertain objects or features of objects, while ignoring other objects(suppressing irrelevant stimuli). Characteristic games suitable for thetraining program 502 include the Odd Beep and Beep Seeker gamesdescribed in WO 2012009117 A1 and WO 2012005953 A1, which are hereinincorporated by reference.

A. Sensory Discrimination

In one embodiment, the training program 2 comprises twenty-five games inthree groups: 1) Odd Out 2) Recall 3) Seeker. All three groups of gameshave at least eight variants, of which four are visual and four areauditory. Visual variants use the following stimuli: (i) orientationlines (ii) shapes of different colors and patterns (iii) directionalmotion and (iv) scenic images. Auditory variants use the followingstimuli: (i) pure tones (ii) sound sweeps (iii) phonemes and (iv) words.These stimuli sets train different hierarchical aspects of the visualand auditory sensory systems. A ninth type of Odd Out variant uses bothauditory and visual stimuli.

The “Odd Out” games train enhancement of a deviant signal amidst aconstant background of distractions. The game challenge increases withcorrect performance as the deviant signal becomes weaker and weaker andstarts blending with the background distractions and becomes moredifficult to discriminate.

The first “Odd Out” variant is “Odd Lines,” which requires theparticipant to discriminate a deviant line pattern on each trial amongsta constant background of lines. The second “Odd Out” variant is “OddShapes,” which requires the participant to discriminate a deviant shapeon each trial amongst a constant background of shapes. Other odd outvariants require the participant to discriminate deviant directions ofmotion, scenes, beeps, sweeps, sounds, and words amongst constantbackgrounds of the same type of stimuli. The “Odd Duo” variant requiresthe participant to discriminate an audiovisual mismatch on each trialamongst a constant background of matched sounds and visuals.

Generally, each “Odd Out” game presents a set of stimuli, all belongingto a common stimulus category, wherein one of the stimuli deviates fromthe rest of the stimuli; and challenges the game participant to indicatewhether any of the patterns displayed was incongruous with the otherpresented stimuli. After receiving the participant's response, the “OddOut” game indicates to the participant whether the participant'sresponse was accurate and repeats the preceding steps over multiplerepetitions. The game adapts to the participant's performance bychanging the number of objects or stimuli and speed at which the objectsor stimuli are presented during the game.

The “Recall” games train suppression of explicit distractions thatappear during the memory period of a delayed working memory task; hencethese games also explicitly train working memory. As the participant'sperformance improves, the game challenge increases by increasing thenumber of distracting signals and decreasing their differentiabilityfrom the remembered targets.

The first “Recall” variant is “Line Recall,” which requires theparticipant to remember a target line pattern while ignoring variousline distractors. The second “Recall” variant is “Shape Recall,” whichrequires the participant to remember a target shape pattern whileignoring various shape distractors. Other “Recall” variants require theparticipant to remember a target motion direction, scene image, beep,sound sweep, phonemic sound, or word while ignoring distractors of thesame type of stimuli.

In general, each “Recall” game presents a first stimulus or set ofstimuli belonging to a stimulus category during a first interval oftime; ceases presentation of the first stimulus or set of stimuli;presents a series of distractor stimuli belonging to the same stimuluscategory during a second interval of time; challenges a game participantto ignore the distractor stimuli; prompts the game participant that achallenge stimulus or set of stimuli is about to be presented; presentsthe challenge stimulus or set of stimuli; and challenges the participantto respond by identifying whether the challenge stimulus or set ofstimuli matches the first stimulus or set of stimuli. After receivingthe participant's response, the game indicates to the participantwhether the participant's response was accurate. The game repeats thepreceding steps over multiple repetitions using progressively longersecond intervals of time. The games adapt to the participant'sperformance by changing the number of objects or stimuli and speed atwhich the objects or stimuli are presented during the game.

The “Seeker” games train suppression of implicit distractions as theparticipant discriminates a pre-defined target from all other stimuli.All stimuli other than the target form the implicit distractions. Thesedistractions are implicit as opposed to explicit, as the participantcontinually decides whether a stimulus is or isn't a distractor. Thelonger the participant can discriminate targets from distractors, thebetter. The game challenge increases with correct performance as thedistracting signals become more similar to the target.

One “Seeker” variant is “Beep Seeker,” which requires the participant todiscriminate a pre-defined target beep in each trial amidst other beepdistractors. Another “Seeker” variant is “Sweep Seeker,” which requiresthe participant to discriminate a pre-defined sound sweep in each trialamidst other sound sweep distractors. Other “Seeker” variants requirethe participant to remember a line, shape, motion direction, sceneimage, phonemic sound, or word while ignoring distractors of the sametype of stimuli.

In general, each “Seeker” game presents a first stimulus or set ofstimuli belonging to a stimulus category during a first interval oftime; ceases presentation of the first stimulus or set of stimuli;presents a second stimulus or set of stimuli belonging to the samestimulus category during a second interval of time; and challenges thegame participant to indicate whether the target stimulus or set ofstimuli was present in the second stimulus or set of stimuli. Afterreceiving the participant's response, the “Seeker” game indicates to theparticipant whether the participant's response was accurate. The“Seeker” games repeat the preceding steps over multiple repetitionsusing progressively longer second intervals of time. The games adapt tothe participant's performance by changing the number of objects orstimuli and speed at which the objects or stimuli are presented duringthe game.

B. Divided Attention Games

In another embodiment, the training program 392 comprises one or moredivided attention games. FIGS. 7 and 8 illustrate two screenshots 121,127 of one embodiment of a divided attention game and game called “TaskSwitcher,” which challenges a game participant to identify whether twostimuli share a target characteristic, wherein the targetcharacteristics change unpredictably.

The game gives the participant a general instruction 122 and a specificrule 123, 128 to follow, like matching objects based on shape or coloror pattern. The game presents two stimuli 124, 125 and challenges theparticipant to indicate—for example, by selecting an appropriate arrowbutton 126—if they match based on the current rule. The game presents anew set of stimuli 129, 130 with each trial. Also, the rules 123, 128change—for example, from “same shape” to “same color”—after a block oftrials or in an unpredictable manner between trials. The participantmust respond quickly, because the trials keep proceeding even withoutparticipant input. The game registers an accurate response with apleasant sound such as a “ding” and an inaccurate response with anunpleasant sound such as a “thunk.” To prod the participant to respondmore quickly, the game presents an indication 131 of the participant'sspeed (which may be a moving average) or speed category.

This cognitive training game and game improves executive function,attention, and working memory. It also improves overall speed andaccuracy of audio and/or visual processing, audio and/or visual workingmemory retention and retrieval, and attention.

It should be noted that the particular game disclosed herein is meant tobe exemplary, and that other repetition-based cognitive training gamesusing audio/visual stimuli with multiple stimulus sets may be used,singly or in combination. The game described herein is but one exampleof a cognitive training game using a computing system to presentaudio/visual stimuli to a participant, record the participant'sresponses, and modify some aspect of the stimuli or its presentationbased on these responses, where these method elements are repeated in aniterative manner using multiple sets of stimuli to improve the executivefunction, attention, and working memory of the participant. Suchcognitive training may include a variety of such audio/visualstimulus-based games, including in a coordinated manner.

B. Sensorimotor Impulse Suppression

In another embodiment, the training program 2 comprises one or moresensorimotor impulse suppression games. FIGS. 9 and 10 illustrate twoscreenshots 132, 136 of a visuospatial information assessment taskcalled “Wango Mango,” which challenges a game participant to press thespacebar whenever an image appears that is not the target, but towithhold pressing the spacebar if the image appears.

Wango Mango displays a target stimulus 135 and examples of a pluralityof foil stimuli 134 and prompts 135 the game participant to respond asquickly as possible (for example, by pressing the space bar) to foilstimuli while refraining from responding to the target stimulus.

VI. WORKING MEMORY GAMES

Each training program 2 also includes a plurality of working memorygames. These games train the participant to remember details inshort-term memory. The details can be visual or auditory stimuli, andthey can relate to objects or people. As the participant improves, thenumber of items to remember increases. The participant may also need tomanipulate the items like reorder them to produce the correct responseon a trial. In embodiments of the training program 2 configured foraddicts, participant-specific distractors may appear within the games.

1. Spatial Memory

FIGS. 11 and 12 illustrate screenshots 138, 143 of one embodiment of avisuospatial working memory game called “Spatial Memory,” whichchallenges a game participant to memorize the locations of severalobjects and thereafter respond if the object is at a rememberedlocation.

Spatial Memory displays a first set of spatially distributedelements—for example, a plurality of balls 141 superimposed on arepresentation of a boundary-marked playing area such as a soccer field142 or sandbox 144—on a physical display for a first time interval.Spatial Memory then clears the first set of spatially distributedelements from the physical display, replacing it with a visual mask.Next, Spatial Memory displays the challenge element 140, 146 on thephysical display for a second interval of time. Spatial Memory prompts145 the participant to indicate whether the challenge element is locatedin the same position as one of the first set of spatially distributedelements. After receiving the participant's response, Spatial Memoryindicates whether the participant's response was accurate. SpatialMemory repeats the preceding steps over multiple repetitions. As theparticipant's performance improves, Spatial Memory uses progressivelylarger first sets of elements and progressively shorter first and secondtime intervals.

2. Brainville

FIGS. 13-24 illustrate screenshots 147, 150, 153, 156, 159, 161, 164,167, 170, 173, 176 and 179 of one embodiment of a sequential visualmemory game called “Brainville,” which challenges a game participant torecall the progression of a journey, the objects seen along thatjourney, and the order in which those objects were seen.

For example, FIG. 13 depicts the journey beginning at a narrator's house149. Brainville notifies 148 the participant that “We are starting frommy house.” Next, Brainville exclaims 151 “Look! A tower!” and displaysan image of the tower 152. This is followed by the statement 154 “Keepwalking,” along with an image 155 of a road and sounds of walking. Next,Brainville exclaims 157 “Look! A surgeon!” together with an image 158 ofa penguin dressed as a surgeon. This is followed by the statement 160“Keep walking,” and the statement 162 “Turn right,” each with an image161, 163 of a straight road or a road with a bend accompanied by soundsof walking. Next, Brainville exclaims 165 “Look! A dead tree!” togetherwith an image 166 of a dead tree. Finally, Brainville announces 168“Hooray! We reached our destination” and displays an image 169 of acastle entrance.

Following this narrative, Brainville quizzes the participant about thejourney. For example, in FIG. 21, Brainville asks 171 “What was myhouse?” and displays a set of answer choices 172 constituting images,one of which is the house 149 displayed at the beginning of the journey.In FIG. 22, Brainville asks 174 “What was the first item?” and displaysanother set of answer choices 175 constituting images, one of which isthe image 158 of a penguin dressed as a surgeon. In FIG. 23, Brainvilleasks 177 “What was the first turn?” and displays a set of answer choices178 constituting images of roads, one of which is straight, one of whichturns left, one of which turns right, and one of which turns a fullhalf-circle. In FIG. 144, Brainville asks 180 “What was thedestination?” constituting images, one of which is the castle entranceimage 169 displayed at the end of the journey.

More generally, Brainville presents a series of stimuli in sequentialfashion, with each stimulus separated by a representation ofprogression, navigation, or traversal away from the previous stimulus.Brainville challenges the game participant to identify one or more ofthe stimuli based upon the order in which the stimulus was presented.Brainville also challenges the game participant to identify acharacteristic of each of at least one of the representations ofprogression away from a stimulus.

3. Visual Memory

FIGS. 25-27 illustrate screenshots 182, 187 and 190 of one embodiment ofa visuospatial memory game called “Visual Memory,” which challenges agame participant to remember objects shown at different spatiallocations across a time delay.

In FIG. 25, Visual Memory prompts 183 the participant that it willbriefly present multiple colors 184, 185 at different locations around acentral object of fixation 186. For example, in FIG. 26, Visual Memorydisplays a purple-colored box 68 to the lower left of a “+” symbol 186and a blue-colored box 189 to the lower right of the “+” symbol 186.After a first time interval, Visual Memory clears the boxes 188, 189from the display, replacing it with a visual mask that lasts for asecond time interval. Then, Visual Memory displays a selectable set ofspatial locations 192 and prompts 191 the participant to indicate thelocation that the blue-colored box 189 had appeared.

The second time interval begins at 1 second and increases as trainingprogresses. Also, as training progresses, the set of possible locationsincreases and details in the objects become more similar. To ensuregeneralizability, the details about the objects change across blocks oftrials, from color hue to overall shapes to minor details in the shape.

4. Musical Games

The training program 2 also includes games that use musically structuredstimuli to integrate auditory, visual and motor sequencing and develophand and voice control. The participant hears a melodic sequence andplays it back from memory by tapping the spacebar, typing theappropriate keys, or singing. Their accuracy is determined by one ormore of three measures: 1) rhythm, 2) duration and 3) pitch.

FIGS. 28-31 illustrate screenshots 193, 195, 198 and 201 of oneembodiment of an auditory and spatial memory game called “Look, Listen,and Repeat,” which challenges a game participant to press buttons hesees and hears in the remembered order. The game is similar to how onelearns to play the piano. The participant hears a musical sequence andplays it back on multiple keyboard keys. The task trains the participantto retain an accurate mental model of the rhythm and pitch of a musicalsequence, and to convert it into an appropriate motor response sequenceby playing it back on multiple keys.

In FIG. 28, the game instructs 194 the participant to place theirfingers on the keyboard as if they are playing the piano, watch andlisten to a series of button presses, and remember and repeat thesequence they saw and heard. FIGS. 29 and 30 illustrate a QWERTYkeyboard portion 197 and two successively highlighted keys 196 and 199.After a predetermined delay, in FIG. 31, the game illustrates the sameQWERTY keyboard portion 197 and prompts 202 the participant to repeatthe series of button presses with the keyboard buttons.

As training increases in difficulty, the sequence of played keys getslonger and is played more quickly. The game also presents more buttonoptions and increases the predetermined delay.

FIGS. 32-35 illustrate screenshots 204, 206, 209 and 212 of oneembodiment of such a game called “Keyboard: Timing,” which challenges agame participant to recall the rhythm of a musical sequence by playingit back on the spacebar. Keyboard: Training trains the participant toretain an accurate mental model of the rhythm of a musical sequence, andto convert it into an appropriate motor response sequence by playing itback on the spacebar.

In FIG. 32, Keyboard: Timing challenges 205 the participant to play backthe melody by tapping a single input device (such as the spacebar) atthe same speed as they hear it. In FIG. 33, Keyboard: Timing displaysmusical notation 207 while playing the melody. In FIG. 34, Keyboard:Timing prompts 210 the participant to play the melody back. Keyboard:Timing also grades the participant's response, provides an indication ofwhether their rhythm and/or pitch was correct, and displays amulti-trial measure 213 of the participant's progress. In oneembodiment, not shown, Keyboard: Timing provides a display thatgraphically compares the participant's response with the correctstimulus sequence.

FIGS. 36 and 37 illustrate screenshots 214, 216 of another embodiment ofa game called “Voice: Timing,” which challenges a game participant torecall the rhythm of a musical sequence by singing the melody back. Theparticipant hears a musical sequence and sings it back. The task trainsthe participant to retain an accurate mental model of the rhythm of amusical sequence, and to convert it into an appropriate vocal motorresponse sequence.

Voice: Timing prompts 215 the participant to sing the melody they hearat the same speed as they hear it. In FIG. 37, Voice: Timing displaysmusical notation 218 and a microphone button 219, while prompting 217the participant to hold the microphone button, sing, and release. Voice:Timing receives the participant's audio input from a microphone, parsesit into a rhythmic sequence, and measures the similarity of theparticipant's timing to that of the played melody.

FIG. 38 illustrates a screenshot 220 of a sequencing and multimodalintegration game called “Voice: Timing & Pitch,” which challenges a gameparticipant to recall the rhythm and pitch of a musical sequence bysinging the melody back. The participant hears a musical sequence andsings it back. The task trains the participant to retain an accuratemental model of the rhythm and pitch of a musical sequence, and toconvert it into an appropriate vocal motor response sequence.Instructions are provided at the beginning of the task.

VII. EXECUTING FUNCTIONING

1. Rule Change

FIGS. 39-41 illustrate screenshots 222, 225 and 227 of one embodiment ofan executive functioning game called “Rule Change,” which challenges agame participant to categorize cards based on conditional rules.

For example, in FIG. 39, Rule Change gives two instructions to the gameparticipant. The first instruction 223 is to respond to a black card ifthe card value is higher or lower than five. The second instruction 224is to respond to a red card if the value is even or odd. In FIG. 40,Rule Change displays a card 226 drawn from a normal desk of cards. Theparticipant is challenged to initially differentiate the card 226 basedon the color of the card (black vs red). Then, in FIG. 41, Rule Changeceases displaying the card 226 and prompts 229-231 the participant tofollow the appropriate rule that involves the number on the card. Theparticipant is challenged to respond as quickly as possible. As trainingprogresses, the rules change more frequently.

2. Mental Rotation

FIGS. 42 and 43 illustrate screenshots 232, 236 of one embodiment ofanother executive functioning game called “Mental Rotation,” whichchallenges a game participant to mentally rotate a letter or number toits right-side up position and indicate whether it is a normal ormirrored image. For example, FIG. 42 illustrates rotated “7” 234 and arotated-and-mirrored “7” 235. Mental Rotation instructs 233 theparticipant to mentally rotate each target image to its up-rightposition and indicate, using keyboard arrows, whether the target imageis a normal or mirror-image. In FIG. 43, the target image 237 is arotated-and-mirrored “4.” The participant is prompted to indicatewhether the target image 237 is a normal or mirrored “4” using the leftarrow key 238 or the right arrow key 239, respectively.

VIII. SELF CONTROL AND REWARD PROCESSING

The training program 2 also includes games 4 that address variousaspects of self-control and reward processing. Games 4 that strengthenself-control utilize techniques like meta-cognition (metric on howimpulsive the user is in comparison to healthy peers who are morepatient), speeding up inhibition processes, manipulation of attention,and control over one's internal state. Games 4 that renormalize rewardvalues train the user to shift their attention towards rewards thathealthy peers consider rewarding (like scenes of family, friends, lifemilestones). More advanced games 4 require the participant to track andmanipulate the rewarding values to perform memory tasks ordecision-making tasks.

Some games 4 train the participant to divert attention away from salientaddiction-related or obsession-triggering stimuli that they should bepaying less attention to, e.g., drugs, fattening food, source of anxietyor negativity (Grin Hunting, Name the Color, The Matrix Recalled). Othergames 4 train through tracking measures of impulsivity in oneself likewarped time perception (Tick Tock) and discounting of future monetaryrewards before and after seeing distracting stimuli (Now or Later).These measures of impulsivity are provided to the participant to gainself-awareness and training tends to focus on metacognition. Aside fromexternal distractions, self-control must also work on internaldistractions. Training to let thoughts and feelings pass by is dealtwith in the mindful breathing exercise (Air Control). Self-control canalso be expressed as poor motor control or “pulling the trigger tooquickly”, so there is one game addressing motor impulsivity (CategoryClick).

In particular, Self-control games train the participant to suppressautomatic processes like reading words, particularly if they relate tosubstances (Stroop task with trigger words), and attending to externalstimuli (Mindfulness task). Participants are also trained to observe andalter their choice behavior to be more consistent with their healthypeers (Temporal choice game that provides self-awareness). Other gamesdeal with time perception and motor control, which are typically lesscontrolled in individuals with SUD. Distractors across the games aredrug-related triggers spanning people, places, things, and actionsassociated with drug use. These distractors are never part of a correctresponse, and so the participant should implicitly shift attention awayfrom them. By having such distractors be irrelevant throughout training,we are dampening the encoding of stimulus-outcome associations in thebrain that pair drug-related triggers with positive outcomes byrendering the positive outcome obsolete.

Reward processing games renormalize reward processing, by having theparticipant focus more on rewards normally considered rewarding, e.g.,family, friends, life milestones, helping others, symbols of happinessand success. The participant not only focuses on such rewards during thegames, but also discovers through implicit training that these rewardslead them to the correct responses. Since reading of social cues isnecessary for deriving reward from them, games in this domain also trainthe participant to notice more details in social cues and accuratelyclassify them based on their emotional properties. Games become moredifficult by requiring the participant to manipulate reward attributesin the task or remembering the attributes against distractors and timedelays. Again, the distractors are tailored to the participant. Byrequiring fast and/or attentive responses to rewarding and emotionalstimuli, these games attempt to strengthen Pavlovian approach respondingtowards people, places, things, or actions normally consideredrewarding.

The stimuli described here are for alcoholism. However, they can bemodified to address other forms of addiction (like food or drugs),anxiety, or negativity.

1. Air Control

FIGS. 44 and 45 illustrate screenshots 240, 242 of one embodiment of amindfulness game called “Air Control,” which challenges a participant topractice mindful breathing. Air Control provides instructions 241 at thebeginning of the task. As shown in screenshot 242, Air Control displaysa peaceful nature scene to facilitate mindfulness. The duration of themindfulness period starts at two minutes and increases by thirty secondsincrements up to five minutes.

2. Grin Hunting

FIGS. 46 and 47 illustrate screenshots 243, 247 of one embodiment of anattentional bias modification game called “Grin Hunting.” In an addict,images, sounds, and smells of objects of their addiction frequentlytrigger craving. Depressed individuals tend to linger on depressingimages and sounds longer than healthy individuals. In PTSD individuals,certain images, sounds, and smells can trigger trauma. Grin Huntingchallenges and trains a game participant—such as an addict or adepressed or traumatized individual—to ignore stimuli that triggerunhealthy psychological responses and/or selectively respond to healthystimuli.

Grin Hunting displays sets of spatially distributed images 244, 245.Each set of images 244, 245 is displayed for a brief time interval,after which the images are cleared from the screen and replaced with asingle image in the position where the image with a positive valence waslocated.

In each set, one of the images—such as of a flower 245 or a smilingface—has a positive valence. The other image in the set tends toinitially trigger or evoke an unhealthy psychological response, such ascraving, anger, or obsession.

In a version or configuration tailored for addicts, Grin Huntingincludes objects—such as a picture 244 of an alcoholic beverage—thattriggers craving in an addict. In a version or configuration tailoredfor depressed individuals, Grin Hunting shows depressing pictures—suchas pictures of a sad facial expression—upon which depressed individualsare more prone to dwell. In a version or configuration tailored fortraumatized individuals, Grin Hunting shows images related to theirtrauma.

The set of positive and negative valence images stimuli remain presentedfor a short duration. Afterwards, they disappear (they are cleared fromthe screen), and a replacement image is displayed in the position wherethe positively valenced image was. The replacement image may have apositive valence, such as a smile, or a neutral valence, such as theimage 248 of an emotionally neutral expression shown in FIG. 47. Thereplacement image is not one that would reinforce the game participant'saddiction, trauma, depression, or vice. If the stimulus contains asmile, the user is to locate and click on the smile to get the trialcorrect. Otherwise, the user waits to continue onto the next trial.

The game receives and times the game participant's selections, providingan indication or measure 246 of the speed of the game participant'sresponse. In a competitive embodiment, Grin Hunting ranks the gameparticipant's selections against those of other game participants afterthe game participant completes a block of trials.

In general, a Grin Hunting trial has two steps, an attention biasingstep and a smile detection step. In the attentional biasing step, two ormore spatially distinct stimuli are presented to the participant, ofwhich one (or a subset) is to be ignored. The rest are all affectivelypositive stimuli. For example, two visual stimuli appear briefly to theleft and right of the center of a digital display (e.g., on a computermonitor, tablet, or phone). One is an image that prompts the undesirablebehavior (for example, the craving for eating or for drinking anaddictive food or beverage). The other is an emotionally positive imagethat does not prompt the undesirable behavior.

The stimuli remain presented for a short duration, then disappear andinitiate the smile detection step. In this step, the previous stimulusnot associated with the undesirable behavior is replaced with anotheremotionally positive image, which may or may not contain a smile in it.If the stimulus contains a smile, the participant is to locate and clickon the smile to get the trial correct. Otherwise, the participant waitsto continue onto the next trial. While the participant performs thetraining, data is collected and analyzed to compute measures of how wellthe participant is in disengaging cues triggering the undesirablebehavior. These measures enable the participant to see regularimprovement, which could increase their motivation to continue withtheir training and behavior modification. Data can also be shared withclinicians and other medical professionals.

Selection of the stimuli is designed for specific behavior modification.Stimuli that draw unwanted exaggerated attention can be used for thosethat are to be ignored in the task and final corrected behavior. Exampleimages would feature cigarettes and lighters for smokers, spider websand daddy longlegs for spider phobics, and bloodshed for soldiers withpost-traumatic stress disorder. Emotionally positive stimuli that do notprompt the behavior to be corrected would feature images of sunsets andtranquil scenery, family and friends, and representations ofinspirational goals and success (which can be personalized). Thosestimuli are then divided for use in the two steps of the trial. Throughnet positive framing of the training stimuli, this cognitive trainingmethod can additionally increase happiness and wellbeing.

Another example of stimuli to be ignored could be images of fatteningfood, from which a dieter trains to divert attention away. The dieterinstead would attend to other stimuli like images of delicious, healthyfoods and positive images of exercise, which predict where thesubsequent image with a smile will be. The task design reduces attentionand value of the stimuli that are to be ignored, while enhancingattention and value of the stimuli that are motivating in correctingbehavior and promoting positive emotions. By focusing on fundamentalmechanisms that guide behavior, Grin Hunting has board applicationsbeyond attention and social cognition training. It can address addictivebehaviors like drug abuse and overeating, depression and other mooddisorders, anxiety and phobias, as well as overall happiness. Further,data collected during the training can serve as assessments of progress.

Eradicating a bad habit requires dissolving neural mechanisms thatencode associations between stimuli that trigger the cascade ofundesirable behavior and actions that make up the undesirable behavior.Meanwhile, it is complemented by strengthening favorable behavior andweakening associates between habit-triggering cues and memories andtheir expectancies and outcomes. The approach of Grin Hunting is todivert attention away from any cue that prompts the undesirablebehavior. For alcoholics, successful training that reduces relapseinvolves disengaging attention towards images related to alcohol.Similar training can be applied to ameliorate other forms of addictionlike smoking, other drugs, or gambling, as well as the treatment ofphobias. Shifts of attention can also be coupled with actions thatantagonize and eventually replace bad habits and abnormal behavior. Forinstance, if a bad habit involves gravitating towards particular items,training would require movements away from said items. Grin Huntingreduces attention to cues that trigger an undesirable behavior, whilemaking them irrelevant to decision-making processes; promotes andrehearses actions that replace the undesirable behavior; and reinforcespositive, pro-social emotions (in the form of smiles) for executing goodbehavior. Instilling positive emotions in the training serves as ananalogue to the release of endorphins after intense physical exercise(i.e., for a dieter) in that good behavior is implicitly reinforced andbroadly represented on a neuronal level in the brain.

In one embodiment, Grin Hunting uses randomized trials, wherein aselected percentage (such as 50%) of the images are followed by smiles.As the participant's accuracy improves or the levels advance, the setsof images are presented at a faster pace, and the presentation durationdecreases. The possible locations of the images also increase as thegame progresses. The stimuli also change in salience or visualproperties like luminance, making diversion of attention away from thecue associated with an undesirable behavior, i.e., image of cigarettesfor breaking the habit of smoking, harder. For example, in a GrinHunting game configured for alcohol addicts, alcohol-related imagesappear progressively brighter and more distracting compared tonon-alcohol-related images. The stimuli also increases in realism, goingfrom cartoon to high-resolution photos to the participant's personalcollection of stimuli. These changes drive bottom-up attentionalprocesses that must then be regulated by top-down attentional processesand strengthened cognitive control. The collective changes affectingattentional bias also scale with how well the participant performs inthe training across trials and days.

While the user performs the training, data is collected and analyzed tocompute measures of how well the user is in disengaging cues triggeringthe undesirable behavior. These measures enable the user to see regularimprovement, which could increase their motivation to continue withtheir training and behavior modification. Data can also be shared withclinicians and other medical professionals.

In one embodiment of an attentional bias modification program, one stageof the Grin Hunting game is used in the assessment phase, and fivestages are used in the training phase.

3. Category Click

FIGS. 48-50 illustrate screenshots 249, 252, and 255 of one embodimentof a game called “Category Click,” which challenges a game participantto selectively respond to stimuli that fall within a target categoryexcept for one or more pre-identified “freeze” stimuli that fall withinthat category.

Category Click presents a target category and a freeze stimulus thatbelongs to that category. FIG. 48 illustrates an instruction 250 thatidentifies the category as “sea life” and illustrates a freeze stimulus251 constituting a stingray image.

After the game participant selects “Start,” Category Click displays asequence of stimuli on the screen. A plurality of the stimuli—forexample, the fish 253 shown in FIG. 49—belong to the target category. Aplurality of the stimuli do not belong to the target category. At leastone of the stimuli includes the freeze stimulus. Furthermore, aplurality of the stimuli—for example, the image 256 of a bottle capbeing removed on FIG. 50—are distractor stimuli, for example, imagesand/or sounds that arouse addictive craving or trigger negativefeelings.

Category Click challenges the game participant to respond to everystimulus that belongs to the category except for the reference stimulus.Category Click also challenges the game participant to withholdproviding any response to any distractor stimuli. For example, theparticipant may be challenged to click a mouse button or press aspacebar every time he or she sees an image within an instructedcategory, but to withhold responding to either the freeze image or anyalcohol-related lures that may appear.

Category Click receives and times the game participant's selections anddisplays a measure 254 of the participant's speed on the screen. At theend of a block of trials, Category Click ranks the game participant'sperformance against those of other game participants.

In one embodiment of an attentional bias modification program, one stageof the Category Click game is used in the assessment phase, and twentystages are used in the training phase. As the game progresses, CategoryClick alternates between various categories of images. Also, as the gameprogresses toward advanced levels, the freeze image becomes harder todifferentiate from the rest of the images in the category.

4. Mood Matchmaker

FIGS. 51-55 illustrate screenshots 258, 261, 264, 267, and 270 of oneembodiment of an affective working memory game called “Mood Matchmaker.”Mood Matchmaker challenges a game participant to match a challengestimulus to a target stimulus in a context that includes a plurality ofdistracting stimuli, including stimuli that trigger unhealthypsychological responses.

This game is also designed to promote empathy, for which severe problemsare noted for individuals with ASD, and to further strengthen ToMelements in the social cognition system. Trainees are required to labelsocial images based on what “the other person” playing the game at thesame time would rate it (the other person is actually a normative ratingobtained from hundreds of image raters). Participants score points onlyif their tags match those of the normative rating.

In this task, Mood Matchmaker presents an image 259 evoking a certainaffective value (emotion/mood)—for example, of a boy smiling whileeating an ice cream cone—and challenges the game participant to rememberit across some time period. During that time period, Mood Matchmakerpresents a sequence of images 262, 265, 268 that can interfere withtheir memory retention.

In embodiments of Mood Matchmaker configured for addicts or depressedindividuals, the sequence of images 262, 265, 268 may include one ormore distracting images 262 or 265 to the game participant. For example,for a game participant struggling with a smoking addiction, thedistractions are related to the game participant's “vice,” such as animage 265 of cigarette smoking that is highly salient to that gameparticipant. For a game participant struggling with depression, thedistractions have a negative affective value, such as a depressing image262 of a shantytown.

After presenting the sequence of distractor images 262, 265, 268, MoodMatchmaker presents a set of images 271, 272 and prompts the participantto pick the one with the most similar affective value as the first image259.

Mood Matchmaker receives and times the game participant's selections anddisplays a measure 134 of the participant's speed on the screen. At theend of a block of trials, Category Click ranks the game participant'sperformance against those of other game participants.

In one embodiment of an attentional bias modification program, one stageof the Mood Matchmaker game is used in the assessment phase, and twelvestages are used in the training phase.

5. Name the Color

FIG. 56 illustrates a screenshot 274 of one embodiment of a game called“Name the Color.” Name that Color challenges a game participant toidentify the font color of words that include words that triggerunhealthy psychological responses.

Name that Color presents a central dot 275 and a plurality of selectablebuttons 276 around the central dot 275, each selectable button 276 beinglabeled with a name of a color, such as “red”, “blue”, “green,” and“black.” The game instructs the game participant to move a cursor overthe dot 275. As soon as the cursor is moved over the dot 275, Name thatColor displays a colored challenge word 277 on the screen.

The challenge word 277 is not a name of the color of the word. Thechallenge words are either neutral words or trigger words related to anaddiction or emotional impairment. For example, in FIG. 56, the neutralchallenge word “rails” is presented in a red-colored font. Examples oftrigger words related to alcoholism include “beer,” “bar,” and “happyhour.” Examples of trigger words related to depression include “bad,”“failure” and “dying.” Name that Color challenges the game participantto ignore the semantic content of the challenge word 277 and select thebutton 276 whose label matches the color of the challenge word 277.

Name that Color measures and tracks the game participant's response timeto each challenge word 277 and provides an indication or measure 269 ofthe game participant's speed. Between some of the trials, Name thatColor re-arranges and/or relabels the selectable buttons, requiring thegame participant to continue reading the words on the buttons 276, whileattempting to ignore the semantic content of the challenge words 277, asthe game goes on.

At the end of a block of trials, Name that Color ranks the gameparticipant's performance against those of other game participants. Inone embodiment of an attentional bias modification program, one stage ofthe Name that Color game is used in the assessment phase, and ten stagesare used in the training phase.

The words come from one of two categories—alcohol-related or neutralwords. The categories alternate across blocks within the task, and theblock size decreases across levels. Colors are red, green, blue, orblack.

6. Now or Later

FIGS. 57 and 58 illustrate screenshots 278, 282 of one embodiment of adelay discounting assessment and training module called “Now or Later.”Now or Later is one example of a game targeting reward processing andself-control, others of which include attention bias modification,mindful breathing, and a daily survey for meta-cognition andself-awareness. An important purpose of such exercises is tosignificantly less craving from baseline.

Now or Later measures a game participant's self-control as indicated bythe participant's degree of craving and willingness to accept delayedgratification. Now or Later initially challenges the game participantwith sets of delay discounting choices between an immediate monetaryreward or a delayed but relatively larger monetary reward. Subsequently,Now or Later presents images of objects of addiction (such as alcoholicbeverages) and asks the game participant to subjectively rate howsignificantly they desire that object. Now or Later then returns to moredelay discounting choices.

For example, in FIG. 57, Now or Later presents a challenge question 279that asks the game participant to select between two monetary rewards,$80 in fourteen days (box 280), or $28 today (box 281). Based on theparticipant's choices across trials, Now or Later approximates howimpulsive the participant is in regards to delaying gratification.

In FIG. 58, Now or Later illustrates an image 284 of an ice bucketfilled with alcoholic beverages, along with the challenge question 283,“Do you want to drink this?” The participant selects a rating 285 of howdesirable the image of the particular “vice” is. Applicants haveobserved that arousal can weaken a game participant's ability to delaygratification. Therefore, by comparing the game participant's choicesbefore and after the vice images, Now or Later assesses the severity ofthe game participant's addiction.

After the game participant has made twenty choices, Now or Laterdisplays a bar graph showing how much self-control the participantexhibited based on his choices during the level. Now or Later thenchallenges the game participant to correctly answer a question relatedto his performance to finish the level.

The first level of the Now or Later game presents images ofnon-alcoholic beverages. Subsequent levels depict images of alcoholicbeverages.

One embodiment of an attentional bias modification program uses the Nowor Later task to select stimuli to incorporate into the otherattentional bias modification games, thereby personalizing the modulesto specific game participants.

Now or Later is useful in assessing and training schizophrenia patients,alcoholics, smokers, methamphetamine abusers, pathological gamblers, andobese individuals, all of whom show higher discount rates in that theyprefer smaller, immediate rewards.

7. Scene That!

FIGS. 59-62 illustrate screenshots 286, 288, 290, and 292 of oneembodiment of a game called “Scene That!” Scene That! challenges a gameparticipant to indicate whether a target stimulus was contained within aset of distracting stimuli, including one or more stimuli that triggerunhealthy psychological responses.

Scene That! briefly presents a target scene 287 and challenges the gameparticipant to remember it. Then, Scene That! presents a stream ofscenes 289, 291 one by one, including highly salient images—such as animage of a large hamburger 289—that stimulates a “vice” such asovereating, alcohol or drug addiction. In some of the trials, SceneThat! also presents the target scene 287 in the stream of images. Afterthe stream of images passes, Scene That!, in screen 292, presents aprompt 293 for the game participant to indicate whether the target scene287 was in the stream.

8. Tick Tock

FIG. 63 illustrates a screenshot 294 of one embodiment of a timetracking game called “Tick Tock,” which displays one or more objects andchallenges the game participant to select the object(s) after theindicated time interval(s). Tick Tock is a time tracking task withn-back and dual-n-back working memory components.

Tick Tock aims to tune up the brain networks involved in time perceptionby training the participant to have more accurate time perception. WhileTick Tock uses visual stimuli, stimuli can be visual, auditory, tactile,or any combination thereof.

Tick Tock first displays a background scene 300. After a brief delay,Tick Tock superimposes images of one or more objects—such as feathers295 and 297—on the background scene 300. Tick Tock also displays one ormore numbers 296, 298 to accompany those objects. Tick Tock alsopresents distractor stimuli 301, which the participant is challenged toignore.

Tick Tock challenges the participant to count the indicated number ofseconds, whenever an image appears paired with a number, and then clickon the image. Tick Tock treats the response as correct if theparticipant-perceived time interval is close enough to the instructedtime interval. That is, if the participant clicks the image within agiven buffer of time around the target time, the participant's responseis treated as correct. The buffer is the allowed temporal error in aresponse to be scored as “correct”.

FIG. 63 shows a trial in which the participant is challenged toconcurrently track two time intervals: four seconds and seven seconds.These intervals may have started at different times in the past. Theparticipant is also challenged to ignore all stimuli that are not pairedwith numbers.

Tick Tock provides the participant with visual and auditory feedback, sothe participant knows if they clicked too soon or too late, or if theyclicked within a given buffer of time around the target time.

As the Tick Tock game progresses into more advanced stages, the numberof time intervals the game participant is required to track increases,as does the set of possible time intervals. Tick Tock advances indifficulty in other dimensions, too, including the number of distractors(images that do not require time tracking), the similarity ofdistractors to target images, the salience of the target images comparedto the background, and the number of possible locations at which thetarget images appear (thus expanding the field of view to which theparticipant must pay attention). Also, as the participant becomes moreaccurate in tracking time intervals, the buffer of time (bounds ofallowed temporal error) around the target time decreases.

In an auditory form of Tick Tock (not shown), instructions, targets, anddistractors are sound clips. The participant may hear something likethis: “In exactly one second after you start hearing this beep <insertbeep sound here>,” click on the button labeled “first intervalcompleted” . . . <beep sound plays now>. Sounds can come from differentlocations if the participant wears binaural headphones. As trainingbecomes harder, the number of targets and distractors increases.Further, targets and distractors will become more similar in sound,while the background becomes noisier. The background starts off silentthough. Each target has its own corresponding button, so that thecomputer can give correct feedback based on which specific time intervalthe participant is responding to. Visual and auditory feedback is givenon the participant's accuracy as described above.

A third, cross-modal version of Tick Tock (also not shown) uses morethan one sensory domain, like combining visual and auditory domains. Inthe case of auditory targets, instructions at the beginning of thesession would dictate what the target sound is. The time interval, onthe other hand, would be set during gameplay from the number that isonscreen when the participant hears the target sound. After the target'stime interval has past, the participant clicks on a button correspondingto which interval had completed, e.g., “first interval completed.”Increasing the difficulty in gameplay uses strategies described above:increase the number of targets and distractors, display visual stimuliacross a larger area, play background sounds simultaneously with targetsounds, and overlay target images on a distracting background (e.g.,gray text on black background; or graphic background with charactersresembling the target, but are not identical to it). The roles of visualand auditory stimuli in this use case can be reversed.

Tactile stimuli can also replace one modality or augment thiscross-modal version of time-tracking-based cognitive training. In thehaptic form, targets and distractors are tactile stimuli likevibrations. A controller device containing the gameplay softwareprovides instructions either visually or aurally. The haptic interfacingcomponent or separate device gets commands from the controller deviceand generates a range of tactile sensations. This component or separatedevice can be either touched or worn by the participant in such a mannerthat responses from the device are easily communicated to theparticipant. The participant indicates when time intervals havecompleted through the controller device or another device thatcommunicates responses back to the controller. If sensations aredelivered as a single point, targets and distractors are differentiatedby frequency and amplitude of vibrations. However, if sensations aredelivered across many spatial positions, then position serves as thethird parameter for differentiating targets and distractors. Increasingthe difficulty in gameplay uses strategies described in the previouscross-modal sample use case. Tactile stimuli can be made more similarand span more spatial locations.

In a retrospective time estimation version of Tick Tock, the participantestimates how much time has past between different events, where one orboth events happened in the past. Feedback is given as to how accuratethe participant's estimate was in the manner described above. Theparticipant sees a background scene, over which images appear. If theimage is a target, at a later point in time, the participant will beasked how much time has past since the target had appeared. There isalso an auditory version of this, where stimuli are sound clips. Ineither version, the participant can be instructed beforehand what thetarget looks like or sounds like. Some images or sounds serve asdistractors and are to be ignored. The first level starts with only onetarget, and the levels progress in difficulty as described above.

In a time reproduction version of Tick Tock, the participant is asked toreproduce time intervals. At the start of a trial, the participant isinformed what the target(s) is. Then during an observation period, oneor more target images appear onscreen and then disappear after differentamount of times. Distractors will do the same thing, but are to beignored. After the observation period ends, the participant is asked toreproduce when, where, and for how long each target image appeared.Visual and auditory feedback about accuracy will be given for eachtarget image. Levels progress in difficulty in the same manner describedin “Use case 1A”. For other use cases, replace visual stimuli withauditory or tactile stimuli, or any combination thereof.

In a time manipulation version of Tick Tock, the participant is asked tomanipulate temporal information like determining which time interval islongest, ranking time intervals from shortest to longest, or matchingtime intervals. At the start of a trial, the participant selects animage and observes how much time it animates for. Then the participantselects a different image and observes how much time it animates for.Afterward, the participant is asked to find the image with the shorteror longer animation. Multiple images can be presented, in which case theparticipant is later asked 1) which image had the shortest or longestanimation, or 2) to rank the images based on their animation durations,or 3) to provide numeric estimates on the animation durations. Visualand auditory feedback is given on the participant's accuracy. Toincrease difficulty of gameplay, these techniques are used: 1) addingmore target images, 2) adding distracting animations that are to beignored, 3) allowing target images to span more onscreen area, and 4)having animation durations become more similar.

In yet another version, the participant sees images one by one, eachappearing for a different amount of time. Then the participant ispresented with all images side by side and asked which one animated forsome inquired number of seconds. The participant clicks on the imagethat they think had appeared for the inquired number of seconds. Visualand auditory feedback is provided as described above. More images can bechained together, and their appearance durations can be made moresimilar to increase task difficulty. Additionally, the images' contentcan be made more distracting by being more emotionally salient and/orcognitively engaging.

In one version of a goal-directed decision-making implementation, theparticipant tries to find an image that will yield the biggest reward.Reward amounts are associated with different time intervals, as are theimages to choose from. Hence to obtain the biggest reward, theparticipant must estimate the animation duration associated with thebiggest reward. Then the participant observes the animation duration ofdifferent images until they find the one whose duration matches that ofthe biggest reward. The training can be made more difficult with morechoice options and more similar animation durations.

In all use cases, visual stimuli can be switched for auditory or tactilestimuli, and cross-modal versions can be implemented. FIG. 64 is afunctional block diagram that summarizes the Tick Tock exercise. Inblock 303, Tick Tock presents time-estimation instructions forresponding to each target stimulus. As discussed in one embodimentabove, the instruction may comprise a number associated with a targetstimulus. Or, as discussed in other embodiments above, the instructionmay comprise being told to wait a number of seconds after a targetstimulus is presented, to estimate the length of time a target stimulusis presented, to estimate the length of time between the presentation oftwo target stimuli, to compare how long different stimuli werepresented, to identify which stimulus was presented the shortest or thelongest, or to rank the stimuli based on the durations they werepresented.

In blocks 304 and 305, Tick Tock presents the target stimulus or targetstimuli and distractor stimulus or distractor stimuli. The targetstimuli may be presented before, after, or at the same time as thedistractor stimuli. In block 306, Tick Tock receives the participant'sresponse(s). If correct, Tick Tock may in step increase a difficultyparameter using the adaptivity procedure discussed earlier by, forexample, decreasing the allowable time estimation error (block),increase the distractor salience (block), increase the number of targetstimuli that the participant is required to track (block), or increasethe field of view (block). If incorrect, Tick Tock may in step maintainor decrease the difficulty level using the adaptivity procedurediscussed earlier.

9. The Matrix Recalled

FIG. 65 illustrates a screenshot 322 of one embodiment of a workingmemory game called “The Matrix Recalled.” The Matrix Recalled presentsan ordered list of items in audible form. In some trials, the orderedlist of items includes a distractor, such as an image of alcohol, thatstimulates an unhealthy psychological response. After presenting thelist, The Matrix Recalled presents a grid 323 and challenges the gameparticipant to recall the items in the requested order.

The participant hears an ordered list of items. Then they must recallthose items in the requested order by clicking on them from a grid. Onoccasion, items in the grid will be distracting images of alcohol. Thedistracting images can be changed to be from other categories.

The Matrix Recalled receives and times the game participant's selectionsand displays a measure 324 of the participant's speed on the screen. Atthe end of a block of trials, Category Click ranks the gameparticipant's performance against those of other game participants.

IX. SOCIAL COGNITION

Another suite of games are designed to assess and train social cognitionabilities, emphasizing improvement in speed of processing of basicsocial cognitive abilities through repetitive practice and implicitlearning. In one embodiment, a suite of games called “SocialVille”provides nineteen different games that collectively target the varioussocial cognition domains of social cue perception, visual and vocalemotion perception, self-referential processing and theory of mind(ToM). The games can be characterized as being one of three types: (1)“speeded” tasks that challenge processing speed by requiring theparticipant to make fast perceptual discriminations; (2) “workingmemory” tasks that require the participant to make working memorymanipulations; and (3) “other” games that do not have a particularemphasis on either processing speed or working memory.

1. Speeded Gaze Direction Identification

FIGS. 66-68 illustrate screenshots 325, 327 and 330 of one embodiment ofa facial cue processing speed game called “Gaze Cast.” The goals of thisgame are to: (1) improve the SC system's ability to track other's gazedirection; (2) strengthen the mirror neuron system by ‘forcing’ thetracking of the gaze direction of others, and; (3) improve the trainee'sability to hold gaze direction in memory.

Each trial of Gaze Cast starts with a central “start” button 326. Oncethe participant clicks on the start button, Gaze Cast presents a videoclip 328 of a person. The person makes a speeded glance shift 329 in oneof many possible directions. Next, the video clip then stops anddisappears. Gaze Cast then presents an array of between two and nineperipheral objects 332 (depending on the level in training) and prompts331 the participant to select the peripheral object in the direction ofthe person's glance.

Gaze Cast provides auditory feedback for both correct and incorrectresponses. The next trial begins three seconds after the participant'sresponse. In Gaze Cast, the duration speed of the video clip isadaptively varied (on a scale from one being slowest to ten beingfastest) based on participant's responses using an up-down adaptivealgorithm. Also, in some embodiments, the speed of the gaze shiftadaptively shortens and the number of peripheral objects adaptivelyincreases as the subject gets better at the task.

2. Speeded Same Gaze Direction Match

FIGS. 69-71 illustrate screenshots 333, 335 and 338 of anotherembodiment of a facial cue processing speed game called “Looky Lou.”Looky Lou presents a target face for a brief interval of time, followedby a visual mask, followed by a subsequent set of faces. Looky Lou thenchallenges the participant to select the face whose eyes are gazing inthe same direction as the target face.

The goal of this game is to improve the subject's implicit ability tofocus attention on the region of the eyes, which has been shown toconvey key social information. Individuals with ASD are known to havesevere difficulties focusing on the eye region or in inferringinformation from the eyes. In this game, Looky Lou presents images ofindividuals looking in various directions, and prompts participants toselect the individual that looks in the same direction as the targetindividual.

Each trial of Looky Lou starts with a central ‘start’ button 334. Oncethe participant clicks on the start button 334, a target face 337 ispresented for a brief period of time. The target face's eyes 336 aredepicted looking at one of nine potential directions (randomly selectedwith equal probability) followed by a visual mask for 500 ms, and thenan array of between two and nine faces 339.

Participants are required to select the face 339 whose eyes are gazingin the same direction as the target face 337 (regardless of faceidentity) by clicking on it with the computer mouse. Auditory feedbackis provided for both correct and incorrect responses, and the next trialbegins three seconds after the participant's response.

In this game, the duration of presentation of the target face 337 isadaptively varied based on participant's responses using a Zestalgorithm, which is a Bayesian adaptive psychometric method that usesmaximum likelihood procedure for threshold estimation. That is, imagesare presented for shorter and shorter time durations (as little as a fewmilliseconds) as subjects' performances improve through a trainingblock. The number of gazes to select from also gets larger asparticipants progress with training.

3. Speeded Facial Image Match

FIGS. 72-74 illustrate screenshots 341, 343 and 345 of anotherembodiment of a facial cue processing speed game called “Face It!” whichpresents a target face from the front, side, or an angle, followed by avisual mask, followed by a subsequent set of faces, and challenges theparticipant to identify the target face in the array.

Impaired processing of faces is one of the consistent findings inindividuals with ASD. The goal of Face It! is to improve the faceperception system's ability to process faces efficiently and to improvethe ability to identify faces viewed from various directions. This formof game heavily engages the Fusiform Face Area (FFA) of the brain, whichhas been shown to be disengaged in ASD. As the participant progresses,Face It! presents the target face for shorter and shorter periods oftime and with fewer peripheral cues (such as hair). Subjectssubsequently need to select the same person (presented from a differentangle) from an array of face choices. In the course of training, thenumber of distractor faces increases, as well as the numbers ofdirections from which images are shown.

Each trial of Face It! starts with a central ‘start’ button 342. Oncethe participant clicks on the start button 342, Face It displays atarget face 344 of a person on a computer screen for a first timeinterval. The target face 344 is presented from either the front, theside, or three-quarters angle. After the first time interval, Face It!clears the image of the target face 344 and presents a visual mask.After a second time interval, Face It! displays an array 346 of facialimages on the computer screen. Face It then prompts 347 the gameparticipant to select the target face 344 from the array 346. Afterreceiving the game participant's response, Face It! provides anindication of whether the game participant's response was correct alongwith an indication or measure of a length of the first time interval.Face It! repeats the preceding steps over a plurality of trials.

To force the participant to memorize key facial characteristics—such asthe look and relative proportions of the eyes, nose, and lips—eachfacial image is cropped. Each image is cropped at the top along theperson's forehead, so that the top of the head is not shown, on thesides along the cheeks so that the ears do not show, and at the bottombelow the mouth so that the chin does not show. The cropping trains theparticipant's attention on the eyes, eyebrows, nose, and mouth of thefacial image.

As the participant's accuracy improves, Face It! progressively reducesthe first time interval. In some advanced trials, the target face in thearray is a rotated or mirrored version of the target image. Adaptivityand threshold calculation is done using 1up-2down procedure, with a stepsize of 50 ms, converging to 71% correct.

4. Speeded Emotional Cue Identification

It is believed that training targeting social-emotional control systems,including explicit facial affect recognition training, will broadlyimprove emotional self-monitoring abilities in stable, chronicschizophrenics. It is believed that training will effectively re-balancedistorted responses to emotionally negative or disturbing stimuli.Training targeting implicit affect attribution (e.g., falselyinterpreting neutral facial expression as sad) should reduce “attentioncapture” effects (i.e., effectively reducing dwell time to sad stimuli).In this form of training, explicit task instructions are orthogonal tothe implicit dimension (i.e., explicit speeded gender discriminationchallenge is accompanied by adapted presentation of emotionalexpressions).

FIGS. 75-77 illustrate screenshots 348, 350 and 352 of one embodiment ofan emotional cue processing speed game called “Name That Feeling.” NameThat Feeling presents a target face 351 expressing an implicit emotion,followed by a visual mask, and challenges the participant to indicatethe emotion that best expresses the emotion exhibited by the targetface.

Name That Feeling's structure is similar to that of Speeded Gaze Castgame described above, except here the target face 351 features anemotion, and the response array 354 is comprised of 2-10 verbal labelsof basic emotions (e.g. ‘happy’, ‘sad’, ‘angry’). Name That Feelingchallenges the game participant to select the emotion that correctlydescribes the emotion presented by the target face. Feedback, adaptivityand threshold calculation are performed similarly to that of the SpeededGaze Match game.

FIGS. 78-80 illustrate screenshots 355, 357 and 359 of anotherembodiment of a processing speed game called “Emotion Motion,” whichpresents a video clip 358 of a face 356 expressing an implicit emotionfollowed by a visual mask. Afterwards, Emotion Motion challenges theparticipant to indicate the emotion that best expresses the emotionexhibited by the target face. Emotion Motion displays a set of words 361representing a variety of emotions and prompts 360 the game participantto select the word that best expresses the emotion of the target face356. After receiving the game participant's response and recording thespeed of that response, Emotion Motion provides an indication of whetherthe game participant's response was correct along with an indication ofa speed of the response.

Emotion Motion is similar in structure to the Name That Feeling gamedescribed above, but instead of using still images of facial emotions,Emotion Motion uses video clips portraying facial emotions. Feedback,adaptivity and threshold calculation are done similarly to the SpeededGaze Direction Identification game described above.

The goals of this game are similar to the ones of the Poke that Feelinggame described below, with the exceptions that: (1) here the affectprocessing is explicit (rather than implicit); (2) the game furtherstrengthens the link between different systems of affect processing byassociating an emotion with a written emotion tag; and (3) the game usesvideo clips of emotions, which have more ecological validity thanstills. Subjects need to decide which emotion is presented in a videoclip.

5. Speeded Face Emotion Match

FIGS. 81-83 illustrate screenshots 362, 364, 366 of one embodiment of aprocessing speed game called “Poke That Feeling.” Poke That Feelingpresents a target image 365 of a face expressing an implicit emotion,followed by a visual mask, followed by a set of facial images 368 eachexpressing a different implicit emotion. Poke That Feeling prompts 367the participant to select a facial image 368 who's expressed emotionbest matches the emotion expressed by the target image 365. Afterreceiving the game participant's response and recording the speed ofthat response, Poke That Feeling provides an indication of whether thegame participant's response was correct along with an indication ormeasure of a length of the first time interval.

The goals of this game are to implicitly strengthen the distributedemotion perception system in the brain, including areas as the amygdala,prefrontal cortex, and superior temporal sulcus that are hypothesized tobe at the core of the social deficits for ASD). The game heavilyexercises this perceptual system by presenting many images of peopleportraying affect.

Poke That Feeling is similar in structure to the Speeded Gaze Matchgame, but here the target face features an emotion and the responsearray features two to eight faces (depending on difficulty), eachshowing a different emotion. As the game participant's performanceimproves, Poke That Feeling reduces the first time interval, requiringthe game participant to identify the implicit emotion more rapidly, and“forcing” the affect perception system to process the relevant affectivefeatures more and more efficiently. Throughout training, the intensityof the emotion gradually decreases and the number of foils in the arrayincreases, forcing the system to uniquely process the attributesrelevant for a given emotion but not others. Feedback, adaptivity, andthreshold calculation are done similarly to the Speeded Gaze Match game.

6. Speeded Valence Match

FIGS. 84-87 illustrate screenshots 369, 371, 373 and 375 of oneembodiment of a processing speed game called “Mass Affect,” whichpresents a target image that has a characteristic valence and challengesthe participant to match it with another image having a similar valence.

Each trial starts with a central ‘start’ button 370. Once theparticipant clicks on the start button 370, Mass Affect briefly presentsan image 372 with a certain valence (which may be rated through asurvey), followed by a plurality of distracting images, such as theimage 374 illustrated in FIG. 86. At the end of the sequence, MassAffect presents a set of images 377, a single one of which has acharacteristic valence that matches the first image 372. Mass Affectprompts 376 the participant to select the image that has a valencematching that of the target image 372. Mass Affect provides auditoryfeedback for both correct and incorrect responses. The duration oftarget image presentation adaptively changes based on the participant'sresponses, using an up-down procedure.

7. Facial Emotion CPT

FIGS. 88-90 illustrate screenshots 378, 382 and 384 of one embodiment ofa processing speed game called “TAPAT MDD,” which presents a series ofsmiling, frowning, and neutral facial images. The game challenges theparticipant to press the spacebar when seeing a smiling or frowningimage but to withhold pressing the spacebar when seeing a neutral image.In so doing, the game trains the participant to quickly recognizeemotionally expressive faces and distinguish them from emotionallyneutral faces.

The game is structured as a tonic and phasic alertness training (TAPAT)task that trains both moment-to-moment and extended aspects ofalertness. Tonic alertness refers to intrinsic arousal that fluctuateson the order of minutes to hours. Tonic alertness is intimately involvedin sustaining attention and provides the cognitive tone for performingcomplicated functions such as working memory and executive control.Phasic alertness, by contrast, is the rapid change in attention due to abrief event and is the basis for operations such as orienting andselective attention.

The game prompts 380 the participant to press the spacebar on his or herkeyboard when shown an image 381 of a smiling (80% occurrence) orfrowning (10% occurrence) face—such as face 383 in FIG. 89—but towithhold such action in response to a neutral face (10% occurrence)—suchas face 385 in FIG. 90. The images in this task are of male and femaleindividuals from children to adults. Inter-stimulus-interval is randomlyselected to be either 600, 1800 or, 3000 ms, with equal probability. Theparticipant is instructed to respond as quickly as possible to anemotional image.

8. Emotion Maintenance

FIGS. 91-94 illustrate screenshots 386, 388, 392 and 395 of oneembodiment of an emotional reversal game called “Bright Whites.” BrightWhites repeatedly challenges the participant to adjust to changingemotional outputs by two characters and identify which character wasmost recently smiling.

The goals of this game are to improve: (1) social cue perception, and(2) the ability to flexibly adapt to a new social rule. The gameutilizes ‘reversal learning’ logic. At the beginning of each trial,Bright Whites presents two neutral faces. The subject is prompted toselect one of them to be their “friend.” Following that selection, oneperson smiles or exhibits an implicitly “nice” expression, while theother becomes angry or remains neutral. The subject should continue toselect the “nice” person, or switch to selecting the “nice” person, asapplicable, until a rule change (after a few trials), and the personthat was previously smiling (for example) now becomes angry.

Turning to FIG. 91, Bright Whites prompts 387 the game participant thatimages of two or more people will be displayed, and to remember whosmiles or who is implicitly being nice. Next, Bright Whites displays aset of neutral target facial images 389, 390 and prompts 391 theparticipant to “pick a friend.” After the participant makes a selection,Bright Whites displays a visual mask followed by a set of target facialimages 393, 394 that it presents simultaneously for a brief timeinterval. No two of the facial images in any given set is of the sameperson. One of the images of each set is of a smiling face or of aperson who is implicitly being nice. Each set is followed by a visualmask such as static, followed by a display of images 397, 398 of thesame faces as shown during the time interval but all having neutralexpressions. Bright White prompts 396 the participant to choose who wassmiling.

After receiving the game participant's response, Bright Whites providesan indication of whether the game participant's response was correctalong with an indication of a length of the time interval. Bright Whiterepeats the preceding steps for multiple trials while progressivelydecreasing the time interval in response to correct identifications.

After five to eight trials, the frowning actor becomes the smilingstimulus and the smiling actor becomes the frowning stimulus. The gameparticipant continues to be prompted to select the smiling face. Thespeed of presentation—that is, the length of the timeinterval—adaptively changes based on the participant's responses using a2down-1up adaptive procedure.

9. Facial Affect Theory of Mind

FIGS. 95-98 illustrate screenshots 399, 401 and 403 of one embodiment ofa facial affect theory of mind game called “What Just Happened?” whichtrains a game participant to apprehend an emotional state of mind. WhatJust Happened? plays short video clips 402 of actors expressing emotionor a neutral face and then prompts 404 the participant to identify whichof multiple options 405, 406 or 407 would best describe an experiencethat elicited the actors' expressed emotions. The speed of presentationadaptively changes based on the participant's responses.

10. Working Memory Emotion Cue

FIGS. 98-101 illustrate screenshots 408, 410, 413 and 416 of oneembodiment of a working memory emotional cue game called “Second ThatEmotion.” Second That Emotion presents an array 411 of down-facing cardsand challenges a participant to select card pairs that show matchingfacial emotions.

In addition to improving the function of the emotion perception system,this game also targets visual spatial working memory for emotions.Participants need to find cards of matching facial expressions in largerand larger card arrays.

During each trial, Second That Emotion presents the participant with anarray 411 of down-facing “cards” and prompts 412 the participant toclick on card pairs that show matching facial emotions. As an example,FIG. 99 illustrates the first card 415, face up, of a selected pair ofcards, and FIG. 100 illustrates the first and second cards 415, 418,face up, of a selected pair of cards. Both cards implicitly revealsadness. When the cards in a selected pair display the same facialemotion—as they do in FIG. 100—Second that Emotion plays a positivefeedback and causes the two cards to disappear.

Second That Emotion challenges the participant to find all matchingpairs in the array within a certain number of clicks (set to be equal tothe number of card pairs) by maintaining representations of facialemotion and of their spatial location in order to guide behavior. If theparticipant matches up all of the correct pairs of facial emotion beforerunning out of clicks, the participant's performance on the trial isconsidered correct. Otherwise, the participant's performance is gradedas incorrect.

The number of card pairs in the array 411 is adaptively varied based onthe participant responses. The number of card pairs is increasedfollowing two consecutive correct trials, and decreased following asingle incorrect trial. The emotions included in this game are: neutral,happy, sad, angry, surprised, afraid, disgusted, proud, contemptuous,pleasantly surprised and embarrassed. In one embodiment, a total oftwenty trials are used, and the threshold (number of card pairs) isdetermined as the arithmetic mean of the last five reversals. Here, areversal refers to the act of flipping over a card to reveal an emotion.Therefore, the last five reversals refers to the last five cards theparticipant flipped over.

11. Working Memory Vocal Emotions

FIGS. 102-104 illustrate screenshots 420, 422 and 425 of an embodimentof a working memory prosody game called “Second That Intonation.” SecondThat Intonation presents an array 423 of down-facing cards associatedwith spoken sentences, and challenges a participant to select card pairsthat match a spoken sentence's prosody with a label.

The goals of this game are similar to the ones in the Second ThatEmotion game, but Second That Intonation uses video clips of emotions(rather than stills), which have more ecological validity and resemblemore everyday stimuli. Subjects are required to find matching pairs ofemotion clips and emotion tags.

This game is similar to Second That Emotion (see above), but requiresmatching pairs of emotion clips and emotion labels rather than stillimages. The face-down sides of half of the cards in the array 426 arelabeled with audio symbol such as a speaker. The other half of the cardsare labeled with a symbol indicating that the card contains a label,such as “happy” or “angry.”

As an example, FIG. 104 illustrates the first card 427, face up, of aselected pair of cards, labeled “happy.” If the second card in theselected pair represents an audio clip of a sentence with a happyprosody, then Second That Intonation plays a positive feedback andcauses the two cards to disappear. Emotions, logic, adaptivity,threshold calculation, and feedback are similar to the Second ThatEmotion game described above.

The emotions included in the Second That Intonation game are: neutral,happy, sad, afraid and angry. In one embodiment, Second That Intonationsamples a set of 100 sentences, all recorded by the same voice-overartist using five different implicit emotions. The maximal size of thearray 423 is ten (five emotion pairs).

Another embodiment of working memory prosody game, called “Voice Match”is similar to “Second That Intonation,” but challenges participants tomatch pairs of spoken sentences, sentence fragments, or words thatexhibit the same prosody. Participants press buttons that are associatedwith different prosodies. Using their auditory working memory to encodethe spatial positions of different prosodies, participants try to pressconsecutive buttons that play identical prosodies. The goals of bothSecond That Intonation and Voice Match are to improve the processing ofprosodic information, and to improve auditory working memory forprosodic inputs.

12. Faces Span

FIGS. 105-111 illustrate screenshots 429, 431, 433, 435, 437, 440 and443 of an embodiment of a working memory facial recognition game called“Face It: Flashback.” Face It: Flashback presents a sequence of faceswith neutral expressions and then challenges the participant to selectthe faces in the order they were presented.

The goals of this game are multifold. In addition to improving the faceperception system's ability to process faces, the goals of this game areto improve the visual face perception system's: (1) temporal modulationtransfer functions and processing speed, and; (2) ability to respond tosuccessively presented face stimuli (memory span abilities for facialstimuli). Participants are required to reconstruct the identity andorder of faces stimuli presented in a sequence, which gets longer andlonger as the subject gets better. The requirement to match faces in thesequence from different angles (frontal, side, profile, ¾ etc.) is alsoadded at later phases of the game, to further boost and automatize thesekey face perception system operations.

FIGS. 106-108 illustrate the game presenting a sequence of side views ofthree faces 432, 434 and 436, all having neutral expressions. Afterbriefly presenting a visual mask, in FIG. 109, Face It: Flashbackpresents all three faces 432, 434 and 436 simultaneously and prompts 439the participant to select the faces in the order they were presented.After the participant selects the first face 432, FIG. 110 shows FaceIt: Flashback presenting the two remaining faces 434 and 436. After theparticipant selects the second face 434, FIG. 111 shows Face It:Flashback presenting the last face 296.

The length of the faces sequence (i.e., the number of faces shown) isadaptively set using a 2up-1down adaptive rule. Each face is presentedfor 1500 ms, with an inter-face interval of 500 ms. One second after thesequence is presented, the same faces appear on the screen, and theparticipant is required to click on them in the order they appeared inthe sequence. The faces used in Face It: Flashback are selected from thesame corpus as that of the “Face It” game.

Similar to FIGS. 105-111, an embodiment of a working memory emotionalcue game is called “Emotion Motion: Flashback.” Emotion Motion:Flashback presents a sequence of videos of faces expressing implicitemotions and then challenges the participant to select the emotions inthe order they were presented.

13. Face Stories Span

FIGS. 112-116 illustrate screenshots 446, 448, 454, 460 and 466 of oneembodiment of a social memory span game called “Face Facts.” Face Factspresents pictures of individuals together with facts about eachindividual and challenges the participant to select facts that are trueabout the individuals.

The goal of this game is to improve working memory for social details,which is problematic in individuals with ASD. On every trial, subjectsare presented with one or more characters and with social detailsassociated with them. Subjects are challenged to memorize those details,as they are later prompted with statements about the face(s) and arechallenged to select the correct ones from the list.

For each trial, Face Facts presents pictures of persons along with threefacts about each person. For example, FIG. 113 illustrates a picture 449of a first person looking into the camera and simulating eye contactwith the game participant. Below the picture 449, Face Facts presentsthree facts 450, 451 and 452 about the first person. Similarly, FIG. 114illustrates a picture 455 of a second person looking into the camera,also simulating eye contact with the game participant. Below the picture455, Face Facts presents three facts 456, 457 and 458 about the secondperson.

Face Facts then presents the faces in random order together with a setof declarative statements, only one of which is consistent with thepreviously presented facts, about that person. Face Facts prompts thegame participant to select the correct statements for each person.

For example, in FIG. 115, Face Facts re-presents the picture 455 of thesecond person along with three declarative statements 447 about her.Face Facts then prompts 459 the game participant to select the statementthat is consistent with the previously presented facts 456, 457 and 458about the second person. Similarly, in FIG. 116, Face Facts re-presentsthe picture 449 of the first person along with three declarativestatements 447 about her. Face Facts then prompts 459 the gameparticipant to select the statement that is consistent with thepreviously presented facts 450, 451 and 452 about the first person.

As the participant progresses through the game, more and more detailsare presented about each character, and the number of characters in thenarrative increases. The length of the sequence (i.e., the number ofindividuals and facts presented) is adaptively set using a 2up-1downadaptive rule.

14. Vocal Emotion ID

FIGS. 117-120 illustrate screenshots 470, 472 and 474 of one embodimentof a prosody apprehension game called “Voice Choice,” which challengesthe game participant to identify the emotion of a neutral sentencespoken with an emotional prosody.

The goals of this game are: (1) to improve the ability of the auditorysystem to discriminate pitch and contour differences in speech relatedto emotions, and (2) to improve the ability of the vocal affectivesystem to process prosodic information across time. Subjects arerequired to detect the emotion expressed in the sentence regardless ofits content, while sentences become shorter and shorter.

Every trial of Voice Choice starts with a presentation of a ‘start’button 471. After clicking it, there is a 1000 ms delay. This isfollowed by a sentence played with neutral content (e.g. ‘Today isTuesday’) but spoken with emotional prosody, such as a happy voice. Theprosody or emotion is unrelated to the content of the sentence. Whilethe sentence is being played, Voice Choice displays a play button 473 ona visually undistracting, uncluttered screen, which helps theparticipant to focus on listening to the story segment.

Next, Voice Choice displays a set of two to five words 475 on thecomputer screen, wherein each word is the name of a different emotion.Voice Choice prompts 476 the game participant to select the word 475that best expresses the emotion of the voice recording. Voice Choicereceives the participant's response and provides an indication ofwhether the participant's response was correct.

The target emotion is randomly selected with equal probability from thefollowing five basic emotions: neutral, happy, sad, angry and afraid.The length of the target sentence is adaptively changed between short(1-2 words), medium (3-4 words) and long (5-7 words) based on a2down-1up adaptive rule, where the sentence gets longer if theparticipant makes a mistake and gets shorter if the participant iscorrect two consecutive times. Threshold is calculated as the arithmeticmean of last five reversals.

15. Social Stories

FIGS. 120-123 illustrate screenshots 477, 479, 481 and 485 of oneembodiment of a social cue apprehension game called “Life Stories,”which challenges game participants to listen to a story and apprehendsocial details in the story.

The goals of this game are: 1) to improve the ability of working memorysystems to maintain and use auditory information in a narrative context;2) to improve the perception of social cues and social nuances; and (3)to improve Theory of Mind (ToM) abilities. In this task, participantsare required to answer questions about key plot elements (both detailsand social elements) of a verbally presented story. The questionsheavily involve ToM aspects by requiring subjects to make socialinferences based on the story.

Life Stories plays segmented stories, each containing multiple socialdetails, to the game participant. Life Stories challenges the gameparticipant to answer questions regarding the story. If the participantanswers more than 80% of the questions correctly, more segments areplayed before the questions appear. Each story has twenty segments and100 questions.

To illustrate, FIG. 121 shows a play button 480 on a visuallyundistracting screen, which helps the participant to focus on listeningto the story segment. After playing the story segment, Life Storiesplays a series of questions such as “As Molly spoke into the phone, whattone of voice did she use?” (FIG. 122) and “What season was it?” (FIG.123). For each question, Life Stories displays a set of answer choices483 or 485, prompting 482 the participant to select the best answerchoice 483 or 485. Life Stories receives the game participant's responseand provides an indication of whether the game participant's responsewas correct.

16. Vocal Affect Theory of Mind

FIGS. 124-126 illustrate screenshots 490, 493 and 496 of one embodimentof a vocal affect theory of mind game called “Say What?” whichchallenges game participants to apprehend a social situation and themeanings conveyed by voice inflection.

This game heavily engages the ToM prosodic system, by requiring subjectsto make judgments about how a person should sound in a given socialsituation, based on the knowledge that they are given about thatsituation. Subjects are presented with social scenarios, and are askedto decide how the characters should sound given the information a) thatthey have about that scenario, and b) given the knowledge that thecharacters have about the scenario.

With each trial, Say What? plays a short script describing a person'scircumstances in a social situation. Say What? then prompts 494 theparticipant with a theory-of-mind question. For example, in FIG. 125,Say What? asks “How might James react in this situation?” Next, SayWhat? presents three aurally-presented response choices 498. Each optionis in the form of the same sentence spoken by the same character, butwith a different prosody and voice inflection (e.g. to reflect angry,excited, or happy mood). Say What? prompts 497 the participant to selectthe sentence with the prosody that best fits the person's circumstancesrelated in the script. The length of the response sentences is variedbased on the participant responses, with longer sentences being playedif the participant makes mistakes.

17. Working Memory Name Span

FIGS. 127-131 illustrate screenshots 500, 502, 505, 508 and 511 of oneembodiment of a name memorization game called “Face and Name,” whichchallenges game participants to associate a plurality of names with aplurality of faces.

In each trial, Face and Name presents pictures of faces 503, 506, 509with names 504, 507, 510. Next, Face and Name randomly selects andpresents previously presented facial images 512, one at a time, with aset of name choices 514, prompting 513 the game participant to selectthe correct name. A trial is deemed successful if the participantcorrectly selects all the names presented. The length of the sequence isadaptively set using a 2up-1down adaptive rule.

18. Auditory Chatter

FIGS. 132-136 illustrate screenshots 516, 518, 520, 522 and 525 of oneembodiment of a vocal emotional cue and theory of mind game called“Auditory Chatter,” which challenges game participants to answerquestions about persons discussed in a social conversation.

As an example, Auditory Chatter presents, as a visual background to thestory, a pencil-sketch-like storyboard rendering 517 of three personsengaged in conversation around a table. Auditory Chatter plays aconversation constituting recordings by voice actors. Auditory Chatterpresents the three persons taking their turns talking about otherpeople—other than the people identified in the rendering 517—that theyknow. The name of each person being talked about is mentioned at leastonce, but preferably only once, in the conversation. As each personspeaks, Auditory Chatter highlights the person 519, 521 to indicate thatthat person is speaking.

After presenting the conversation, Auditory Chatter prompts 523 the gameparticipant with questions 523, 526 about persons discussed in theconversation. The participant is challenged to select from a set ofanswer choices 524, 527.

19. Social Theory of Mind

FIGS. 137-141 illustrate screenshots 529, 531, 533, 539 and 541 of oneembodiment of a theory-of-mind game called “Social Scenes,” whichpresents a short written narrative about a person and challenges theparticipant to infer what that person would have thought or felt basedon the narrated circumstances.

Social Scene taps into neural ToM mechanisms, by helping subjectspractice more and more complex scenarios that require ToM inferences,ranging from first- to second- to third-level ToM. Social Scene presentsparticipants with more and more complex social scenes featuring at leasttwo people, and challenges participants to figure out the knowledge thateach of the characters has, then infer, based on this knowledge, whatthat character will do next.

Social Scenes displays a short written narrative describing a socialsituation. After the game participant hits “Enter” or selects the “OK”button, Social Scenes presents a social question 534 about thenarrative, such as “how is John feeling right now?” Social Scenes alsopresents a set of response choices 535, 536 and 537, and prompts 538 theparticipant to select the response choices in the order from most toleast likely. Each time the participant makes a correct selection, thecorresponding choice disappears from the display, as illustrated in theprogression from FIGS. 139 to 141.

20. What's Joe Thinking

FIGS. 142-144 illustrate screenshots of one embodiment of a theory ofmind game called “What's Joe Thinking?” which challenges gameparticipants to follow the direction of different people's eye gazes andinterpret those gazes as denoting their thoughts.

What's Joe Thinking? displays, for a first time interval, a plurality offacial images 544-545 or 560-568 and plurality of objects 546-547 or552-559 on a display area. A first target facial image 544, 568 is of afirst person (for, example, “Joe”) whose eyes are directed toward atargeted one 546, 556 of the plurality of objects. A second targetfacial image 545, 563 is approximately located along the direction ofthe first person's eyes. The second target facial image 545, 563 is of asecond person whose eyes are either directed toward or away from thetarget object 546. Next, What's Joe Thinking? clears the display area,briefly displays a visual mask, and then displays a set of objectchoices 572 on the display area.

In a first challenge, What's Joe Thinking? prompts 571 the gameparticipant to select the object choice 572 that matches the targetobject 546, 556. After receiving the participant's response to the firstchallenge, What's Joe Thinking? prompts the game participant to indicatewhether the second person's eyes were directed toward or away from thetarget object 546, 556.

What's Joe Thinking? provides an indication of whether the gameparticipant's responses were correct along with an indication or measureof a length of the first time interval. What's Joe Thinking? alsoprogressively reduces the first time interval as the game participant'saccuracy improves. In a typical trial, What's Joe Thinking? presents thetarget facial image 568 in the middle of a matrix of facial images560-568 and objects 552-559, challenging the game participant toidentify one of eight possible directions in which the target image'sgaze is directed.

X. CONCLUSION

Those skilled in the art should appreciate that they can readily use thedisclosed conception and specific embodiments as a basis for designingor modifying other structures for carrying out the same purposes of thepresent invention without departing from the spirit and scope of theinvention as defined by the appended claims. The detailed descriptiondescribes several distinct training programs, schemas, games, andcategories of cognition, stimuli, and disorders. It will be understoodthat not all of that, while exemplary, not all of that detail isessential to the claimed invention. Training programs that employdifferent schemas, provide different games, or that cover more or fewercategories of stimuli and cognition may also be effective in treatingany given disorder. Moreover, it will be understood that the inventiondisclosed herein may well have applications to other cognitive deficitsand disorders.

The invention claimed is:
 1. A training program comprising a combinationof automated games configured to systematically drive neurologicalchanges to treat major depressive disorder (MDD) by treating differentcognitive impairments associated with MDD, the training programcomprising: an inference renormalization game configured to: reveal tothe game participant a category of target stimuli; present a pluralityof stimuli on an electronic display, including a freeze stimulus thatbelongs the category, a second subset of stimuli other than the freezestimulus that belongs to the category, and a third subset of distractorstimuli that do not belong to the category, wherein at least some of thedistractor stimuli are negatively affective; and prompt a gameparticipant to respond through a game piece to only the second subset ofstimuli and to refrain from responding to both the distractor stimuliand the freeze stimulus; receive the game participant's input throughthe game piece; provide an indication to the game participant of whetherthe game participant's input was accurate; and repeat the presentingthrough providing an indication steps over multiple repetitions whileadapting one or more difficulty parameters to target maintenance of asuccess rate within a predetermined range; whereby the gameparticipant's efforts to refrain from responding to both thecategory-related freeze stimulus and the non-category-related distractorstimuli, including negatively affective stimuli, and concentratedefforts to respond to category stimuli other than the freeze stimulus,treatment of the game participant's MDD is effected.
 2. The trainingprogram of claim 1, wherein the negatively affective stimuli are stimulithat express a negative emotion, and the positively affective stimuliare stimuli that express a positive emotion.
 3. The training program ofclaim 2, wherein the negatively and positively affective stimuli arefacial expressions and/or vocal prosodies.
 4. The training program ofclaim 1, wherein the distractor stimuli include stimuli to which thegame participant is prone toward an unhealthy psychological response,including depression.
 5. The training program of claim 1, furthercomprising progressively increasing the distractibility of thenegatively affective stimuli.
 6. The training program of claim 1,wherein the category is a category of living things and the freezestimulus is a living thing that belongs to the category.
 7. The trainingprogram of claim 1, further comprising a processing speed game thatserves visual and/or auditory stimuli, wherein the processing speed gameis effective in upregulating activities in cerebral attention andneuromodulatory control networks that are impaired in a game participantdiagnosed with MDD.
 8. The training program of claim 7, wherein theprocessing speed game presents the centrally presented visual stimuli ata progressively faster rate as the game progresses.
 9. The trainingprogram of claim 7, wherein the processing speed game presents auditorysound sweeps to the game participant.
 10. A training program comprisinga combination of automated games configured to systematically driveneurological changes to treat major depressive disorder (MDD) bytreating different cognitive impairments associated with MDD, thetraining program comprising: an inference renormalization gameconfigured to: present a plurality of stimuli on an electronic display,including a first subset of the stimuli that are positively affective, asecond subset of the stimuli that are negatively affective, and a thirdsubset of which are neutral; and prompt a game participant to respondaccording to a first mode when presented with positively or negativelyaffective stimuli, and according to a second mode when presented withneutral stimuli, wherein one of the first and second modes comprisesresponding through a game piece to a stimulus, and another of the firstand second modes comprises refraining from responding through the gamepiece to the stimulus; receive the game participant's input through thegame piece; provide an indication to the game participant of whether thegame participant's input was accurate; and repeat the presenting throughproviding an indication steps over multiple repetitions while adaptingone or more difficulty parameters to target maintenance of a successrate within a predetermined range.
 11. The training program of claim 10,wherein the first subset of stimuli include smiling images and thesecond subset of stimuli include frowning faces.
 12. The trainingprogram of claim 10, wherein the negatively affective stimuli are vocalprosodies that express a negative emotion, and the positively affectivestimuli are vocal prosodies that express a positive emotion.
 13. Thetraining program of claim 10, wherein the negatively affective stimuliinclude stimuli to which the game participant is prone toward anunhealthy psychological response, including depression.
 14. The trainingprogram of claim 10, further comprising progressively increasing thedistractibility of the negatively affective stimuli.
 15. The trainingprogram of claim 10, further comprising a processing speed game thatserves visual and/or auditory stimuli, wherein the processing speed gameis effective in upregulating activities in cerebral attention andneuromodulatory control networks that are impaired in a game participantdiagnosed with MDD.
 16. The training program of claim 15, wherein theprocessing speed game presents the centrally presented visual stimuli ata progressively faster rate as the game progresses.
 17. The trainingprogram of claim 15, wherein the processing speed game presents auditorysound sweeps to the game participant.
 18. The training program of claim10, wherein the inference renormalization game is configured to: stoppresenting the plurality of stimuli after presenting the plurality ofstimuli for a brief time interval, and after stopping presentment of theplurality of stimuli, present a replacement stimulus; and prompt thegame participant to selectively respond to the replacement stimulus whenit is a positively affective stimulus and to selectively withholdresponding to the replacement stimulus when it is a negatively affectivestimulus.